python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4741 -l 30 ' -s datou_current_4741 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3404830 load datou : 4741 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5732, 'mask_refus_amiens_050123', 16384, 25088, 'mask_refus_amiens_050123', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 1, 5, 13, 23, 59), datetime.datetime(2023, 1, 5, 13, 23, 59)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5972, 'learn_entrant_syctomXV_111023', 16384, 25088, 'learn_entrant_syctomXV_111023', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 10, 11, 15, 57, 35), datetime.datetime(2023, 10, 11, 15, 57, 35)) load thcls load THCL from format json or kwargs add thcl : 3453 in CacheModelConfig load THCL from format json or kwargs add thcl : 3783 in CacheModelConfig load pdts add pdt : 5732 in CacheModelConfig add pdt : 5972 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4741, datou_cur_ids : ['3623726'] with mtr_portfolio_ids : ['26290978'] and first list_photo_ids : [] new path : /proc/3404830/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, mask_detect, blur_detection, brightness, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , BFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 3 ; length of list_pids : 3 ; length of list_args : 3 time to download the photos : 0.6466968059539795 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Thu Aug 28 11:41:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10603 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-08-28 11:41:32.259471: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-08-28 11:41:32.292483: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-08-28 11:41:32.294519: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f4b2c000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-08-28 11:41:32.294550: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-08-28 11:41:32.298696: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-08-28 11:41:32.463284: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x20ac1780 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-08-28 11:41:32.463349: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-08-28 11:41:32.464628: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-28 11:41:32.465054: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-28 11:41:32.467566: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-28 11:41:32.470198: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-28 11:41:32.470661: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-28 11:41:32.473535: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-28 11:41:32.474954: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-28 11:41:32.480766: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-28 11:41:32.482722: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-28 11:41:32.482875: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-28 11:41:32.483858: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-08-28 11:41:32.483878: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-08-28 11:41:32.483889: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-08-28 11:41:32.485643: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9821 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-08-28 11:41:32.783771: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-28 11:41:32.783902: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-28 11:41:32.783925: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-28 11:41:32.783947: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-28 11:41:32.783967: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-28 11:41:32.783987: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-28 11:41:32.784008: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-28 11:41:32.784028: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-28 11:41:32.785681: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-28 11:41:32.787314: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-28 11:41:32.787352: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-28 11:41:32.787371: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-28 11:41:32.787388: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-28 11:41:32.787428: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-28 11:41:32.787446: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-28 11:41:32.787463: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-28 11:41:32.787480: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-28 11:41:32.789173: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-28 11:41:32.789223: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-08-28 11:41:32.789234: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-08-28 11:41:32.789244: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-08-28 11:41:32.790694: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9821 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl3783 thcls : [{'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5972 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5972, 'learn_entrant_syctomXV_111023', 16384, 25088, 'learn_entrant_syctomXV_111023', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 10, 11, 15, 57, 35), datetime.datetime(2023, 10, 11, 15, 57, 35)) {'thcl': {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'sac', 'textiles', 'verre', 'organique', 'dasri', 'masque', 'encombrant', 'autre_emballage', 'autre_non_emballage', 'environnement'], 'list_hashtags_csv': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'svm_hashtag_type_desc': 5972, 'photo_desc_type': 5972, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'sac', 'textiles', 'verre', 'organique', 'dasri', 'masque', 'encombrant', 'autre_emballage', 'autre_non_emballage', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_entrant_syctomXV_111023 NUM_CLASSES 11 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_entrant_syctomXV_111023 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-08-28 11:41:45.358224: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-28 11:41:45.529760: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_entrant_syctomXV_111023 /data/models_weight/learn_entrant_syctomXV_111023/mask_model.h5 size_local : 256052544 size in s3 : 256052544 create time local : 2023-10-26 16:36:43 create time in s3 : 2023-10-11 13:57:17 mask_model.h5 already exist and didn't need to update list_images length : 3 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 13 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 4 Detection mask done ! Trying to reset tf kernel 3405044 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1862 tf kernel not reseted sub process len(results) : 3 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 3 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 7065 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl3783 Catched exception ! Connect or reconnect ! thcls : [{'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5972 ['background', 'sac', 'textiles', 'verre', 'organique', 'dasri', 'masque', 'encombrant', 'autre_emballage', 'autre_non_emballage', 'environnement'] time for calcul the mask position with numpy : 0.0005555152893066406 nb_pixel_total : 15074 time to create 1 rle with old method : 0.01779913902282715 length of segment : 209 time for calcul the mask position with numpy : 0.0022826194763183594 nb_pixel_total : 98201 time to create 1 rle with old method : 0.1090230941772461 length of segment : 627 time for calcul the mask position with numpy : 0.0002675056457519531 nb_pixel_total : 10165 time to create 1 rle with old method : 0.011750459671020508 length of segment : 96 time for calcul the mask position with numpy : 0.0009629726409912109 nb_pixel_total : 68616 time to create 1 rle with old method : 0.07703018188476562 length of segment : 307 time for calcul the mask position with numpy : 0.0033142566680908203 nb_pixel_total : 193114 time to create 1 rle with new method : 0.01160883903503418 length of segment : 539 time for calcul the mask position with numpy : 0.0006678104400634766 nb_pixel_total : 29245 time to create 1 rle with old method : 0.03429555892944336 length of segment : 238 time for calcul the mask position with numpy : 0.0005183219909667969 nb_pixel_total : 42075 time to create 1 rle with old method : 0.04729056358337402 length of segment : 239 time for calcul the mask position with numpy : 0.0005867481231689453 nb_pixel_total : 25523 time to create 1 rle with old method : 0.029897689819335938 length of segment : 208 time for calcul the mask position with numpy : 0.0006368160247802734 nb_pixel_total : 42608 time to create 1 rle with old method : 0.04846453666687012 length of segment : 251 time for calcul the mask position with numpy : 1.1273062229156494 nb_pixel_total : 559715 time to create 1 rle with new method : 0.049652814865112305 length of segment : 744 time for calcul the mask position with numpy : 0.007485628128051758 nb_pixel_total : 359687 time to create 1 rle with new method : 0.023473024368286133 length of segment : 711 time for calcul the mask position with numpy : 0.007750511169433594 nb_pixel_total : 412273 time to create 1 rle with new method : 0.021392345428466797 length of segment : 1143 time for calcul the mask position with numpy : 0.001123666763305664 nb_pixel_total : 40938 time to create 1 rle with old method : 0.04569602012634277 length of segment : 246 time for calcul the mask position with numpy : 0.0021364688873291016 nb_pixel_total : 102467 time to create 1 rle with old method : 0.11604523658752441 length of segment : 283 time for calcul the mask position with numpy : 0.0022978782653808594 nb_pixel_total : 135173 time to create 1 rle with old method : 0.16753339767456055 length of segment : 613 time for calcul the mask position with numpy : 0.0011489391326904297 nb_pixel_total : 72255 time to create 1 rle with old method : 0.08199763298034668 length of segment : 269 time for calcul the mask position with numpy : 0.0037920475006103516 nb_pixel_total : 165204 time to create 1 rle with new method : 0.009249448776245117 length of segment : 457 time for calcul the mask position with numpy : 0.001134634017944336 nb_pixel_total : 80177 time to create 1 rle with old method : 0.09206795692443848 length of segment : 313 time for calcul the mask position with numpy : 0.003678560256958008 nb_pixel_total : 157092 time to create 1 rle with new method : 0.005698680877685547 length of segment : 400 time for calcul the mask position with numpy : 0.0005438327789306641 nb_pixel_total : 15832 time to create 1 rle with old method : 0.018245220184326172 length of segment : 135 time for calcul the mask position with numpy : 0.006865739822387695 nb_pixel_total : 491638 time to create 1 rle with new method : 0.02982497215270996 length of segment : 527 time for calcul the mask position with numpy : 0.009159564971923828 nb_pixel_total : 435212 time to create 1 rle with new method : 0.026103973388671875 length of segment : 911 time for calcul the mask position with numpy : 1.1380505561828613 nb_pixel_total : 5940162 time to create 1 rle with new method : 0.779888391494751 length of segment : 5425 time for calcul the mask position with numpy : 0.0004909038543701172 nb_pixel_total : 33492 time to create 1 rle with old method : 0.041924476623535156 length of segment : 86 time for calcul the mask position with numpy : 0.08452224731445312 nb_pixel_total : 2413662 time to create 1 rle with new method : 0.390413761138916 length of segment : 3395 time for calcul the mask position with numpy : 0.5568861961364746 nb_pixel_total : 7197598 time to create 1 rle with new method : 0.7421858310699463 length of segment : 2618 time spent for convertir_results : 11.270797729492188 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 26 chid ids of type : 4854 Number RLEs to save : 0 save missing photos in datou_result : time spend for datou_step_exec : 44.84819412231445 time spend to save output : 0.09069514274597168 total time spend for step 1 : 44.938889265060425 step2:mask_detect Thu Aug 28 11:42:14 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 4502 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-08-28 11:42:16.860919: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-08-28 11:42:16.884641: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-08-28 11:42:16.886666: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f4b38000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-08-28 11:42:16.886729: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-08-28 11:42:16.890175: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-08-28 11:42:16.992469: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x215ad3e0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-08-28 11:42:16.992511: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-08-28 11:42:16.993677: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-28 11:42:16.994051: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-28 11:42:16.996792: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-28 11:42:17.002505: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-28 11:42:17.002865: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-28 11:42:17.006014: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-28 11:42:17.007530: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-28 11:42:17.012931: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-28 11:42:17.014114: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-28 11:42:17.014175: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-28 11:42:17.014781: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-08-28 11:42:17.014795: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-08-28 11:42:17.014804: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-08-28 11:42:17.015886: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6061 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-08-28 11:42:17.123225: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-28 11:42:17.123309: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-28 11:42:17.123332: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-28 11:42:17.123353: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-28 11:42:17.123373: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-28 11:42:17.123393: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-28 11:42:17.123426: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-28 11:42:17.123448: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-28 11:42:17.124780: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-28 11:42:17.125908: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-28 11:42:17.125945: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-28 11:42:17.125966: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-28 11:42:17.125986: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-28 11:42:17.126005: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-28 11:42:17.126024: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-28 11:42:17.126043: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-28 11:42:17.126063: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-28 11:42:17.127360: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-28 11:42:17.127389: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-08-28 11:42:17.127400: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-08-28 11:42:17.127409: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-08-28 11:42:17.128799: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6061 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl3453 thcls : [{'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5732 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5732, 'mask_refus_amiens_050123', 16384, 25088, 'mask_refus_amiens_050123', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 1, 5, 13, 23, 59), datetime.datetime(2023, 1, 5, 13, 23, 59)) {'thcl': {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'svm_hashtag_type_desc': 5732, 'photo_desc_type': 5732, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME mask_refus_amiens_050123 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : mask_refus_amiens_050123 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-08-28 11:42:27.520312: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-28 11:42:27.698391: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/mask_refus_amiens_050123 /data/models_weight/mask_refus_amiens_050123/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2023-01-05 13:44:07 create time in s3 : 2023-01-05 12:23:52 mask_model.h5 already exist and didn't need to update list_images length : 3 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 60 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 59 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 6 Detection mask done ! Trying to reset tf kernel 3407629 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1311 tf kernel not reseted sub process len(results) : 3 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 3 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 6600 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl3453 Catched exception ! Connect or reconnect ! thcls : [{'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5732 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.00779271125793457 nb_pixel_total : 171353 time to create 1 rle with new method : 0.01273202896118164 length of segment : 317 time for calcul the mask position with numpy : 0.030086040496826172 nb_pixel_total : 409816 time to create 1 rle with new method : 0.0812983512878418 length of segment : 1262 time for calcul the mask position with numpy : 0.0015299320220947266 nb_pixel_total : 43926 time to create 1 rle with old method : 0.0512392520904541 length of segment : 236 time for calcul the mask position with numpy : 0.002635478973388672 nb_pixel_total : 113238 time to create 1 rle with old method : 0.1317427158355713 length of segment : 395 time for calcul the mask position with numpy : 0.28583216667175293 nb_pixel_total : 2428739 time to create 1 rle with new method : 0.7213242053985596 length of segment : 2209 time for calcul the mask position with numpy : 0.0022792816162109375 nb_pixel_total : 91121 time to create 1 rle with old method : 0.10126566886901855 length of segment : 581 time for calcul the mask position with numpy : 0.005784273147583008 nb_pixel_total : 332630 time to create 1 rle with new method : 0.013273954391479492 length of segment : 880 time for calcul the mask position with numpy : 0.0015270709991455078 nb_pixel_total : 68614 time to create 1 rle with old method : 0.07474565505981445 length of segment : 450 time for calcul the mask position with numpy : 0.0005142688751220703 nb_pixel_total : 15425 time to create 1 rle with old method : 0.01857280731201172 length of segment : 117 time for calcul the mask position with numpy : 0.0036356449127197266 nb_pixel_total : 150374 time to create 1 rle with new method : 0.008694171905517578 length of segment : 724 time for calcul the mask position with numpy : 0.009968996047973633 nb_pixel_total : 336137 time to create 1 rle with new method : 0.030701398849487305 length of segment : 1296 time for calcul the mask position with numpy : 0.004193544387817383 nb_pixel_total : 131259 time to create 1 rle with old method : 0.14103126525878906 length of segment : 569 time for calcul the mask position with numpy : 0.010652303695678711 nb_pixel_total : 281078 time to create 1 rle with new method : 0.017486095428466797 length of segment : 969 time for calcul the mask position with numpy : 0.002626180648803711 nb_pixel_total : 163062 time to create 1 rle with new method : 0.007575511932373047 length of segment : 737 time for calcul the mask position with numpy : 0.004064083099365234 nb_pixel_total : 228368 time to create 1 rle with new method : 0.011260032653808594 length of segment : 570 time for calcul the mask position with numpy : 0.002275705337524414 nb_pixel_total : 97055 time to create 1 rle with old method : 0.10489439964294434 length of segment : 436 time for calcul the mask position with numpy : 0.0010502338409423828 nb_pixel_total : 46103 time to create 1 rle with old method : 0.05122661590576172 length of segment : 189 time for calcul the mask position with numpy : 0.006476402282714844 nb_pixel_total : 225538 time to create 1 rle with new method : 0.012319087982177734 length of segment : 551 time for calcul the mask position with numpy : 0.0009260177612304688 nb_pixel_total : 28457 time to create 1 rle with old method : 0.04477739334106445 length of segment : 211 time for calcul the mask position with numpy : 0.0008649826049804688 nb_pixel_total : 43592 time to create 1 rle with old method : 0.05234575271606445 length of segment : 378 time for calcul the mask position with numpy : 0.0023076534271240234 nb_pixel_total : 92707 time to create 1 rle with old method : 0.10733580589294434 length of segment : 379 time for calcul the mask position with numpy : 0.002981424331665039 nb_pixel_total : 143551 time to create 1 rle with old method : 0.15862417221069336 length of segment : 359 time for calcul the mask position with numpy : 0.0030014514923095703 nb_pixel_total : 58020 time to create 1 rle with old method : 0.06822538375854492 length of segment : 475 time for calcul the mask position with numpy : 0.004944562911987305 nb_pixel_total : 156086 time to create 1 rle with new method : 0.014685630798339844 length of segment : 497 time for calcul the mask position with numpy : 0.001132965087890625 nb_pixel_total : 55203 time to create 1 rle with old method : 0.07248830795288086 length of segment : 266 time for calcul the mask position with numpy : 0.007317543029785156 nb_pixel_total : 134186 time to create 1 rle with old method : 0.15311837196350098 length of segment : 728 time for calcul the mask position with numpy : 0.0247342586517334 nb_pixel_total : 707221 time to create 1 rle with new method : 0.0796046257019043 length of segment : 1044 time for calcul the mask position with numpy : 0.001575469970703125 nb_pixel_total : 75942 time to create 1 rle with old method : 0.08625221252441406 length of segment : 252 time for calcul the mask position with numpy : 0.015572071075439453 nb_pixel_total : 557529 time to create 1 rle with new method : 0.18639349937438965 length of segment : 765 time for calcul the mask position with numpy : 0.001478433609008789 nb_pixel_total : 65496 time to create 1 rle with old method : 0.07249832153320312 length of segment : 240 time for calcul the mask position with numpy : 0.007592916488647461 nb_pixel_total : 377595 time to create 1 rle with new method : 0.031180858612060547 length of segment : 1035 time for calcul the mask position with numpy : 0.0022580623626708984 nb_pixel_total : 65427 time to create 1 rle with old method : 0.07203125953674316 length of segment : 535 time for calcul the mask position with numpy : 0.0021102428436279297 nb_pixel_total : 76760 time to create 1 rle with old method : 0.08630943298339844 length of segment : 465 time for calcul the mask position with numpy : 0.0011336803436279297 nb_pixel_total : 64628 time to create 1 rle with old method : 0.07163023948669434 length of segment : 233 time for calcul the mask position with numpy : 0.0031664371490478516 nb_pixel_total : 153535 time to create 1 rle with new method : 0.008877038955688477 length of segment : 607 time for calcul the mask position with numpy : 0.0008132457733154297 nb_pixel_total : 25868 time to create 1 rle with old method : 0.0288543701171875 length of segment : 217 time for calcul the mask position with numpy : 0.010312318801879883 nb_pixel_total : 249429 time to create 1 rle with new method : 0.021019697189331055 length of segment : 1240 time for calcul the mask position with numpy : 0.0053141117095947266 nb_pixel_total : 218894 time to create 1 rle with new method : 0.010872602462768555 length of segment : 440 time for calcul the mask position with numpy : 0.0012285709381103516 nb_pixel_total : 57058 time to create 1 rle with old method : 0.06623053550720215 length of segment : 491 time for calcul the mask position with numpy : 0.00122833251953125 nb_pixel_total : 56385 time to create 1 rle with old method : 0.06304001808166504 length of segment : 293 time for calcul the mask position with numpy : 0.0012700557708740234 nb_pixel_total : 69402 time to create 1 rle with old method : 0.07862687110900879 length of segment : 360 time for calcul the mask position with numpy : 0.002367734909057617 nb_pixel_total : 122649 time to create 1 rle with old method : 0.13537836074829102 length of segment : 467 time for calcul the mask position with numpy : 0.0029048919677734375 nb_pixel_total : 81979 time to create 1 rle with old method : 0.09270024299621582 length of segment : 685 time for calcul the mask position with numpy : 0.014038562774658203 nb_pixel_total : 394465 time to create 1 rle with new method : 0.03993964195251465 length of segment : 522 time for calcul the mask position with numpy : 0.0011696815490722656 nb_pixel_total : 48164 time to create 1 rle with old method : 0.05649566650390625 length of segment : 296 time for calcul the mask position with numpy : 0.0012164115905761719 nb_pixel_total : 57830 time to create 1 rle with old method : 0.07984066009521484 length of segment : 306 time for calcul the mask position with numpy : 0.0013015270233154297 nb_pixel_total : 85026 time to create 1 rle with old method : 0.10499286651611328 length of segment : 214 time for calcul the mask position with numpy : 0.0008716583251953125 nb_pixel_total : 30423 time to create 1 rle with old method : 0.03677845001220703 length of segment : 273 time for calcul the mask position with numpy : 0.00278472900390625 nb_pixel_total : 127473 time to create 1 rle with old method : 0.14814138412475586 length of segment : 235 time for calcul the mask position with numpy : 0.014866113662719727 nb_pixel_total : 609759 time to create 1 rle with new method : 0.055227041244506836 length of segment : 807 time for calcul the mask position with numpy : 0.010045051574707031 nb_pixel_total : 228886 time to create 1 rle with new method : 0.01747751235961914 length of segment : 913 time for calcul the mask position with numpy : 0.012277603149414062 nb_pixel_total : 460732 time to create 1 rle with new method : 0.03178858757019043 length of segment : 1114 time for calcul the mask position with numpy : 0.0013341903686523438 nb_pixel_total : 64316 time to create 1 rle with old method : 0.07411861419677734 length of segment : 300 time for calcul the mask position with numpy : 0.0008323192596435547 nb_pixel_total : 40985 time to create 1 rle with old method : 0.04909396171569824 length of segment : 225 time for calcul the mask position with numpy : 0.0032889842987060547 nb_pixel_total : 160950 time to create 1 rle with new method : 0.010417938232421875 length of segment : 601 time for calcul the mask position with numpy : 0.003703594207763672 nb_pixel_total : 147425 time to create 1 rle with old method : 0.17945075035095215 length of segment : 659 time for calcul the mask position with numpy : 0.0026826858520507812 nb_pixel_total : 136045 time to create 1 rle with old method : 0.14846587181091309 length of segment : 637 time for calcul the mask position with numpy : 0.0011947154998779297 nb_pixel_total : 59287 time to create 1 rle with old method : 0.07002401351928711 length of segment : 276 time for calcul the mask position with numpy : 0.003682374954223633 nb_pixel_total : 154467 time to create 1 rle with new method : 0.010794401168823242 length of segment : 573 time for calcul the mask position with numpy : 0.01889491081237793 nb_pixel_total : 661959 time to create 1 rle with new method : 0.07525849342346191 length of segment : 943 time for calcul the mask position with numpy : 0.0023055076599121094 nb_pixel_total : 121633 time to create 1 rle with old method : 0.13129258155822754 length of segment : 570 time for calcul the mask position with numpy : 0.0022444725036621094 nb_pixel_total : 105761 time to create 1 rle with old method : 0.11775898933410645 length of segment : 363 time for calcul the mask position with numpy : 0.007559061050415039 nb_pixel_total : 310735 time to create 1 rle with new method : 0.03081989288330078 length of segment : 607 time for calcul the mask position with numpy : 0.0007042884826660156 nb_pixel_total : 30176 time to create 1 rle with old method : 0.0337834358215332 length of segment : 151 time for calcul the mask position with numpy : 0.0004627704620361328 nb_pixel_total : 22599 time to create 1 rle with old method : 0.025289535522460938 length of segment : 163 time for calcul the mask position with numpy : 0.0018835067749023438 nb_pixel_total : 76315 time to create 1 rle with old method : 0.0866849422454834 length of segment : 316 time for calcul the mask position with numpy : 0.0024170875549316406 nb_pixel_total : 98242 time to create 1 rle with old method : 0.10881900787353516 length of segment : 478 time for calcul the mask position with numpy : 0.004387617111206055 nb_pixel_total : 211036 time to create 1 rle with new method : 0.01003885269165039 length of segment : 482 time for calcul the mask position with numpy : 0.0019109249114990234 nb_pixel_total : 73438 time to create 1 rle with old method : 0.08257699012756348 length of segment : 310 time for calcul the mask position with numpy : 0.012982845306396484 nb_pixel_total : 598518 time to create 1 rle with new method : 0.03972125053405762 length of segment : 1009 time for calcul the mask position with numpy : 0.013336896896362305 nb_pixel_total : 652111 time to create 1 rle with new method : 0.05389881134033203 length of segment : 855 time for calcul the mask position with numpy : 0.0017011165618896484 nb_pixel_total : 89985 time to create 1 rle with old method : 0.10713410377502441 length of segment : 344 time for calcul the mask position with numpy : 0.004416227340698242 nb_pixel_total : 206909 time to create 1 rle with new method : 0.011204242706298828 length of segment : 500 time for calcul the mask position with numpy : 0.006450653076171875 nb_pixel_total : 400090 time to create 1 rle with new method : 0.01384735107421875 length of segment : 576 time for calcul the mask position with numpy : 0.004721879959106445 nb_pixel_total : 220658 time to create 1 rle with new method : 0.012232542037963867 length of segment : 436 time for calcul the mask position with numpy : 0.005164623260498047 nb_pixel_total : 164869 time to create 1 rle with new method : 0.013160943984985352 length of segment : 385 time for calcul the mask position with numpy : 0.0019233226776123047 nb_pixel_total : 79707 time to create 1 rle with old method : 0.09180402755737305 length of segment : 285 time for calcul the mask position with numpy : 0.003924369812011719 nb_pixel_total : 158665 time to create 1 rle with new method : 0.010751008987426758 length of segment : 485 time for calcul the mask position with numpy : 0.002640247344970703 nb_pixel_total : 85116 time to create 1 rle with old method : 0.09650087356567383 length of segment : 396 time for calcul the mask position with numpy : 0.0010209083557128906 nb_pixel_total : 38855 time to create 1 rle with old method : 0.04480767250061035 length of segment : 273 time for calcul the mask position with numpy : 0.002288341522216797 nb_pixel_total : 73515 time to create 1 rle with old method : 0.1095876693725586 length of segment : 295 time for calcul the mask position with numpy : 0.0035834312438964844 nb_pixel_total : 161305 time to create 1 rle with new method : 0.011049747467041016 length of segment : 394 time for calcul the mask position with numpy : 0.006204128265380859 nb_pixel_total : 227233 time to create 1 rle with new method : 0.017016887664794922 length of segment : 382 time for calcul the mask position with numpy : 0.0007982254028320312 nb_pixel_total : 40144 time to create 1 rle with old method : 0.045096397399902344 length of segment : 260 time for calcul the mask position with numpy : 0.0023746490478515625 nb_pixel_total : 103925 time to create 1 rle with old method : 0.11825370788574219 length of segment : 516 time for calcul the mask position with numpy : 0.006721973419189453 nb_pixel_total : 188039 time to create 1 rle with new method : 0.019495010375976562 length of segment : 828 time for calcul the mask position with numpy : 0.0027189254760742188 nb_pixel_total : 112911 time to create 1 rle with old method : 0.1519472599029541 length of segment : 487 time for calcul the mask position with numpy : 0.0011954307556152344 nb_pixel_total : 60374 time to create 1 rle with old method : 0.07131791114807129 length of segment : 299 time for calcul the mask position with numpy : 0.0003941059112548828 nb_pixel_total : 21048 time to create 1 rle with old method : 0.02492380142211914 length of segment : 95 time for calcul the mask position with numpy : 0.0006103515625 nb_pixel_total : 19979 time to create 1 rle with old method : 0.02309274673461914 length of segment : 326 time for calcul the mask position with numpy : 0.015421628952026367 nb_pixel_total : 585028 time to create 1 rle with new method : 0.0911550521850586 length of segment : 1496 time for calcul the mask position with numpy : 0.002391815185546875 nb_pixel_total : 152788 time to create 1 rle with new method : 0.005941867828369141 length of segment : 506 time for calcul the mask position with numpy : 0.006884336471557617 nb_pixel_total : 392698 time to create 1 rle with new method : 0.014738798141479492 length of segment : 519 time for calcul the mask position with numpy : 0.0007011890411376953 nb_pixel_total : 14401 time to create 1 rle with old method : 0.016455411911010742 length of segment : 169 time for calcul the mask position with numpy : 0.0009517669677734375 nb_pixel_total : 40079 time to create 1 rle with old method : 0.045188188552856445 length of segment : 333 time for calcul the mask position with numpy : 0.0008580684661865234 nb_pixel_total : 33487 time to create 1 rle with old method : 0.037264347076416016 length of segment : 220 time for calcul the mask position with numpy : 0.0009684562683105469 nb_pixel_total : 50934 time to create 1 rle with old method : 0.057701826095581055 length of segment : 207 time for calcul the mask position with numpy : 0.0014026165008544922 nb_pixel_total : 59462 time to create 1 rle with old method : 0.06682491302490234 length of segment : 250 time for calcul the mask position with numpy : 0.00044345855712890625 nb_pixel_total : 28059 time to create 1 rle with old method : 0.03180050849914551 length of segment : 166 time for calcul the mask position with numpy : 0.0007798671722412109 nb_pixel_total : 34908 time to create 1 rle with old method : 0.0398557186126709 length of segment : 173 time for calcul the mask position with numpy : 0.0007219314575195312 nb_pixel_total : 28897 time to create 1 rle with old method : 0.03346538543701172 length of segment : 206 time for calcul the mask position with numpy : 0.0012149810791015625 nb_pixel_total : 59001 time to create 1 rle with old method : 0.06758332252502441 length of segment : 260 time for calcul the mask position with numpy : 0.009847164154052734 nb_pixel_total : 519869 time to create 1 rle with new method : 0.3900151252746582 length of segment : 964 time for calcul the mask position with numpy : 0.002170085906982422 nb_pixel_total : 115268 time to create 1 rle with old method : 0.1288738250732422 length of segment : 493 time for calcul the mask position with numpy : 0.0010876655578613281 nb_pixel_total : 48499 time to create 1 rle with old method : 0.05432438850402832 length of segment : 226 time for calcul the mask position with numpy : 0.002332925796508789 nb_pixel_total : 96772 time to create 1 rle with old method : 0.10807371139526367 length of segment : 355 time for calcul the mask position with numpy : 0.004819393157958984 nb_pixel_total : 254110 time to create 1 rle with new method : 0.01266932487487793 length of segment : 618 time for calcul the mask position with numpy : 0.009474992752075195 nb_pixel_total : 536407 time to create 1 rle with new method : 0.020067691802978516 length of segment : 573 time for calcul the mask position with numpy : 0.003975868225097656 nb_pixel_total : 154738 time to create 1 rle with new method : 0.011065006256103516 length of segment : 715 time for calcul the mask position with numpy : 0.005297183990478516 nb_pixel_total : 326780 time to create 1 rle with new method : 0.012256622314453125 length of segment : 636 time for calcul the mask position with numpy : 0.004274129867553711 nb_pixel_total : 171278 time to create 1 rle with new method : 0.012516975402832031 length of segment : 1169 time for calcul the mask position with numpy : 0.0028502941131591797 nb_pixel_total : 136169 time to create 1 rle with old method : 0.16219687461853027 length of segment : 730 time for calcul the mask position with numpy : 0.012708187103271484 nb_pixel_total : 531821 time to create 1 rle with new method : 0.06953620910644531 length of segment : 1061 time for calcul the mask position with numpy : 0.024378538131713867 nb_pixel_total : 796914 time to create 1 rle with new method : 0.3890814781188965 length of segment : 1738 time for calcul the mask position with numpy : 0.0026552677154541016 nb_pixel_total : 142928 time to create 1 rle with old method : 0.1621112823486328 length of segment : 335 time for calcul the mask position with numpy : 0.006098508834838867 nb_pixel_total : 292016 time to create 1 rle with new method : 0.017320871353149414 length of segment : 612 time for calcul the mask position with numpy : 0.022917985916137695 nb_pixel_total : 741279 time to create 1 rle with new method : 0.1505424976348877 length of segment : 1485 time for calcul the mask position with numpy : 0.0021545886993408203 nb_pixel_total : 94612 time to create 1 rle with old method : 0.10860419273376465 length of segment : 358 time for calcul the mask position with numpy : 0.0029075145721435547 nb_pixel_total : 112343 time to create 1 rle with old method : 0.12579965591430664 length of segment : 643 time for calcul the mask position with numpy : 0.5960242748260498 nb_pixel_total : 6979013 time to create 1 rle with new method : 0.9055209159851074 length of segment : 6021 time for calcul the mask position with numpy : 0.004271507263183594 nb_pixel_total : 284009 time to create 1 rle with new method : 0.011737346649169922 length of segment : 756 time for calcul the mask position with numpy : 0.0008575916290283203 nb_pixel_total : 58302 time to create 1 rle with old method : 0.06392383575439453 length of segment : 275 time for calcul the mask position with numpy : 0.004282474517822266 nb_pixel_total : 353380 time to create 1 rle with new method : 0.009775876998901367 length of segment : 691 time for calcul the mask position with numpy : 0.01264500617980957 nb_pixel_total : 296211 time to create 1 rle with new method : 0.2052750587463379 length of segment : 1005 time for calcul the mask position with numpy : 0.017035961151123047 nb_pixel_total : 1403492 time to create 1 rle with new method : 0.22133827209472656 length of segment : 748 time spent for convertir_results : 28.031871557235718 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 151 chid ids of type : 4854 +++++++++++++++++++++++++++++++++Number RLEs to save : 0 save missing photos in datou_result : time spend for datou_step_exec : 79.13765287399292 time spend to save output : 0.4788632392883301 total time spend for step 2 : 79.61651611328125 step3:blur_detection Thu Aug 28 11:43:33 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: variance laplacian (4160, 3120) 991.1598989500137 (4160, 3120) 1172.3631311147594 (4160, 3120) 276.4735240769179 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 3 time used for this insertion : 0.016094446182250977 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 3 time used for this insertion : 0.01678180694580078 save missing photos in datou_result : time spend for datou_step_exec : 2.045308828353882 time spend to save output : 0.0375819206237793 total time spend for step 3 : 2.082890748977661 step4:brightness Thu Aug 28 11:43:35 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1756374088_3404830_1379844701_1250cca4c1146ec6eb5d48d93a6e4719.jpg treat image : temp/1756374088_3404830_1379844407_615d6d2a885a58566fbbc0964e8e8f9c.jpg treat image : temp/1756374088_3404830_1379844391_ae7bedb41ca1608654673f4a2d4570d9.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 3 time used for this insertion : 0.013846158981323242 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 3 time used for this insertion : 0.04763174057006836 save missing photos in datou_result : time spend for datou_step_exec : 4.591930150985718 time spend to save output : 0.06636595726013184 total time spend for step 4 : 4.65829610824585 step5:rle_unique_nms_with_priority Thu Aug 28 11:43:40 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 151 chid ids of type : 4854 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 68 nb_hashtags : 8 time to prepare the origin masks : 75.71659517288208 time for calcul the mask position with numpy : 0.6053593158721924 nb_pixel_total : 3241265 time to create 1 rle with new method : 1.3262889385223389 time for calcul the mask position with numpy : 0.054472923278808594 nb_pixel_total : 127001 time to create 1 rle with old method : 0.1369633674621582 time for calcul the mask position with numpy : 0.05691170692443848 nb_pixel_total : 284030 time to create 1 rle with new method : 0.9866499900817871 time for calcul the mask position with numpy : 0.053134918212890625 nb_pixel_total : 43778 time to create 1 rle with old method : 0.047350406646728516 time for calcul the mask position with numpy : 0.05430269241333008 nb_pixel_total : 665 time to create 1 rle with old method : 0.0009298324584960938 time for calcul the mask position with numpy : 0.057912349700927734 nb_pixel_total : 2678 time to create 1 rle with old method : 0.005570650100708008 time for calcul the mask position with numpy : 0.13517260551452637 nb_pixel_total : 689740 time to create 1 rle with new method : 1.2222392559051514 time for calcul the mask position with numpy : 0.05570387840270996 nb_pixel_total : 71770 time to create 1 rle with old method : 0.08635878562927246 time for calcul the mask position with numpy : 0.05104184150695801 nb_pixel_total : 6350 time to create 1 rle with old method : 0.00743865966796875 time for calcul the mask position with numpy : 0.051579952239990234 nb_pixel_total : 75737 time to create 1 rle with old method : 0.08171534538269043 time for calcul the mask position with numpy : 0.05112147331237793 nb_pixel_total : 68494 time to create 1 rle with old method : 0.07917523384094238 time for calcul the mask position with numpy : 0.053812503814697266 nb_pixel_total : 9988 time to create 1 rle with old method : 0.011133193969726562 time for calcul the mask position with numpy : 0.05777764320373535 nb_pixel_total : 224888 time to create 1 rle with new method : 1.164292573928833 time for calcul the mask position with numpy : 0.0586247444152832 nb_pixel_total : 57810 time to create 1 rle with old method : 0.06518912315368652 time for calcul the mask position with numpy : 0.051645517349243164 nb_pixel_total : 68561 time to create 1 rle with old method : 0.07348155975341797 time for calcul the mask position with numpy : 0.05120086669921875 nb_pixel_total : 221591 time to create 1 rle with new method : 1.1216228008270264 time for calcul the mask position with numpy : 0.05597853660583496 nb_pixel_total : 378771 time to create 1 rle with new method : 1.0432286262512207 time for calcul the mask position with numpy : 0.05462336540222168 nb_pixel_total : 15021 time to create 1 rle with old method : 0.015737533569335938 time for calcul the mask position with numpy : 0.053704261779785156 nb_pixel_total : 409887 time to create 1 rle with new method : 1.3830089569091797 time for calcul the mask position with numpy : 0.05538010597229004 nb_pixel_total : 4923 time to create 1 rle with old method : 0.005668163299560547 time for calcul the mask position with numpy : 0.0547490119934082 nb_pixel_total : 2534 time to create 1 rle with old method : 0.0031135082244873047 time for calcul the mask position with numpy : 0.05823206901550293 nb_pixel_total : 151304 time to create 1 rle with new method : 1.0046718120574951 time for calcul the mask position with numpy : 0.05223393440246582 nb_pixel_total : 493 time to create 1 rle with old method : 0.0006334781646728516 time for calcul the mask position with numpy : 0.052341461181640625 nb_pixel_total : 28285 time to create 1 rle with old method : 0.031203031539916992 time for calcul the mask position with numpy : 0.05459856986999512 nb_pixel_total : 1514 time to create 1 rle with old method : 0.0020341873168945312 time for calcul the mask position with numpy : 0.05328702926635742 nb_pixel_total : 143933 time to create 1 rle with old method : 0.15288233757019043 time for calcul the mask position with numpy : 0.050759077072143555 nb_pixel_total : 19915 time to create 1 rle with old method : 0.02142333984375 time for calcul the mask position with numpy : 0.05156135559082031 nb_pixel_total : 131038 time to create 1 rle with old method : 0.14222240447998047 time for calcul the mask position with numpy : 0.05346345901489258 nb_pixel_total : 218293 time to create 1 rle with new method : 1.208280324935913 time for calcul the mask position with numpy : 0.053095340728759766 nb_pixel_total : 96941 time to create 1 rle with old method : 0.10441374778747559 time for calcul the mask position with numpy : 0.050975799560546875 nb_pixel_total : 150599 time to create 1 rle with new method : 1.2246928215026855 time for calcul the mask position with numpy : 0.05176591873168945 nb_pixel_total : 52377 time to create 1 rle with old method : 0.05482363700866699 time for calcul the mask position with numpy : 0.05331897735595703 nb_pixel_total : 25888 time to create 1 rle with old method : 0.02733135223388672 time for calcul the mask position with numpy : 0.05197572708129883 nb_pixel_total : 92371 time to create 1 rle with old method : 0.10080194473266602 time for calcul the mask position with numpy : 0.05163288116455078 nb_pixel_total : 84618 time to create 1 rle with old method : 0.08966779708862305 time for calcul the mask position with numpy : 0.05080413818359375 nb_pixel_total : 271 time to create 1 rle with old method : 0.0003647804260253906 time for calcul the mask position with numpy : 0.05096149444580078 nb_pixel_total : 143111 time to create 1 rle with old method : 0.15036797523498535 time for calcul the mask position with numpy : 0.05408811569213867 nb_pixel_total : 1806 time to create 1 rle with old method : 0.0023610591888427734 time for calcul the mask position with numpy : 0.052126169204711914 nb_pixel_total : 63851 time to create 1 rle with old method : 0.0662081241607666 time for calcul the mask position with numpy : 0.05261588096618652 nb_pixel_total : 56163 time to create 1 rle with old method : 0.05968785285949707 time for calcul the mask position with numpy : 0.17487597465515137 nb_pixel_total : 2421564 time to create 1 rle with new method : 1.2846226692199707 time for calcul the mask position with numpy : 0.05373811721801758 nb_pixel_total : 276604 time to create 1 rle with new method : 1.2881112098693848 time for calcul the mask position with numpy : 0.0525667667388916 nb_pixel_total : 1259 time to create 1 rle with old method : 0.0016329288482666016 time for calcul the mask position with numpy : 0.05695199966430664 nb_pixel_total : 65288 time to create 1 rle with old method : 0.06868743896484375 time for calcul the mask position with numpy : 0.05304360389709473 nb_pixel_total : 30183 time to create 1 rle with old method : 0.032030344009399414 time for calcul the mask position with numpy : 0.05191922187805176 nb_pixel_total : 182885 time to create 1 rle with new method : 1.3954296112060547 time for calcul the mask position with numpy : 0.053292274475097656 nb_pixel_total : 3090 time to create 1 rle with old method : 0.003644227981567383 time for calcul the mask position with numpy : 0.05443906784057617 nb_pixel_total : 26333 time to create 1 rle with old method : 0.029695510864257812 time for calcul the mask position with numpy : 0.053034067153930664 nb_pixel_total : 81222 time to create 1 rle with old method : 0.0872948169708252 time for calcul the mask position with numpy : 0.05292701721191406 nb_pixel_total : 122542 time to create 1 rle with old method : 0.13243770599365234 time for calcul the mask position with numpy : 0.05311274528503418 nb_pixel_total : 54427 time to create 1 rle with old method : 0.058242082595825195 time for calcul the mask position with numpy : 0.05362534523010254 nb_pixel_total : 155668 time to create 1 rle with new method : 1.1566576957702637 time for calcul the mask position with numpy : 0.05443692207336426 nb_pixel_total : 69050 time to create 1 rle with old method : 0.07429838180541992 time for calcul the mask position with numpy : 0.05487060546875 nb_pixel_total : 65699 time to create 1 rle with old method : 0.07131052017211914 time for calcul the mask position with numpy : 0.052829742431640625 nb_pixel_total : 22123 time to create 1 rle with old method : 0.024506092071533203 time for calcul the mask position with numpy : 0.0561976432800293 nb_pixel_total : 334867 time to create 1 rle with new method : 1.070016860961914 time for calcul the mask position with numpy : 0.05226922035217285 nb_pixel_total : 15619 time to create 1 rle with old method : 0.017360687255859375 time for calcul the mask position with numpy : 0.054529428482055664 nb_pixel_total : 50037 time to create 1 rle with old method : 0.05679821968078613 time for calcul the mask position with numpy : 0.05686473846435547 nb_pixel_total : 331998 time to create 1 rle with new method : 1.1913022994995117 time for calcul the mask position with numpy : 0.05665397644042969 nb_pixel_total : 408624 time to create 1 rle with new method : 0.9940207004547119 time for calcul the mask position with numpy : 0.052832841873168945 nb_pixel_total : 157973 time to create 1 rle with new method : 2.0803747177124023 time for calcul the mask position with numpy : 0.05494213104248047 nb_pixel_total : 227947 time to create 1 rle with new method : 1.191155195236206 time for calcul the mask position with numpy : 0.05330228805541992 nb_pixel_total : 61888 time to create 1 rle with old method : 0.07289528846740723 time for calcul the mask position with numpy : 0.05945324897766113 nb_pixel_total : 113067 time to create 1 rle with old method : 0.12527871131896973 time for calcul the mask position with numpy : 0.056452035903930664 nb_pixel_total : 170810 time to create 1 rle with new method : 1.131321907043457 time for calcul the mask position with numpy : 0.055136680603027344 nb_pixel_total : 483 time to create 1 rle with old method : 0.0011856555938720703 time for calcul the mask position with numpy : 0.06050920486450195 nb_pixel_total : 17255 time to create 1 rle with old method : 0.01959228515625 time for calcul the mask position with numpy : 0.056757450103759766 nb_pixel_total : 27083 time to create 1 rle with old method : 0.031237363815307617 time for calcul the mask position with numpy : 0.05566596984863281 nb_pixel_total : 15359 time to create 1 rle with old method : 0.017337560653686523 create new chi : 32.897390842437744 time to delete rle : 0.04217195510864258 batch 1 Loaded 69 chid ids of type : 4855 Number RLEs to save : 59581 TO DO : save crop sub photo not yet done ! save time : 7.889081716537476 nb_obj : 70 nb_hashtags : 9 time to prepare the origin masks : 76.47665548324585 time for calcul the mask position with numpy : 0.6647729873657227 nb_pixel_total : 4165612 time to create 1 rle with new method : 1.23655104637146 time for calcul the mask position with numpy : 0.05390524864196777 nb_pixel_total : 23090 time to create 1 rle with old method : 0.025829076766967773 time for calcul the mask position with numpy : 0.05672478675842285 nb_pixel_total : 359027 time to create 1 rle with new method : 1.0775439739227295 time for calcul the mask position with numpy : 0.053585052490234375 nb_pixel_total : 413180 time to create 1 rle with new method : 1.235910177230835 time for calcul the mask position with numpy : 0.051448822021484375 nb_pixel_total : 2809 time to create 1 rle with old method : 0.003171205520629883 time for calcul the mask position with numpy : 0.051035165786743164 nb_pixel_total : 105 time to create 1 rle with old method : 0.00016164779663085938 time for calcul the mask position with numpy : 0.05287361145019531 nb_pixel_total : 201 time to create 1 rle with old method : 0.0005822181701660156 time for calcul the mask position with numpy : 0.05462837219238281 nb_pixel_total : 33357 time to create 1 rle with old method : 0.036599159240722656 time for calcul the mask position with numpy : 0.0540776252746582 nb_pixel_total : 48349 time to create 1 rle with old method : 0.06131100654602051 time for calcul the mask position with numpy : 0.06002163887023926 nb_pixel_total : 22549 time to create 1 rle with old method : 0.02630138397216797 time for calcul the mask position with numpy : 0.05218076705932617 nb_pixel_total : 28790 time to create 1 rle with old method : 0.0330805778503418 time for calcul the mask position with numpy : 0.053522586822509766 nb_pixel_total : 77071 time to create 1 rle with old method : 0.08225417137145996 time for calcul the mask position with numpy : 0.053643226623535156 nb_pixel_total : 160962 time to create 1 rle with new method : 1.3737828731536865 time for calcul the mask position with numpy : 0.052286624908447266 nb_pixel_total : 2718 time to create 1 rle with old method : 0.002961874008178711 time for calcul the mask position with numpy : 0.0569157600402832 nb_pixel_total : 454942 time to create 1 rle with new method : 1.2983739376068115 time for calcul the mask position with numpy : 0.06151604652404785 nb_pixel_total : 310190 time to create 1 rle with new method : 1.260084867477417 time for calcul the mask position with numpy : 0.05373191833496094 nb_pixel_total : 17331 time to create 1 rle with old method : 0.019109487533569336 time for calcul the mask position with numpy : 0.053185224533081055 nb_pixel_total : 20539 time to create 1 rle with old method : 0.02277398109436035 time for calcul the mask position with numpy : 0.05425143241882324 nb_pixel_total : 15766 time to create 1 rle with old method : 0.017473697662353516 time for calcul the mask position with numpy : 0.05402803421020508 nb_pixel_total : 148 time to create 1 rle with old method : 0.0003597736358642578 time for calcul the mask position with numpy : 0.05456948280334473 nb_pixel_total : 104019 time to create 1 rle with old method : 0.11063361167907715 time for calcul the mask position with numpy : 0.05264735221862793 nb_pixel_total : 60273 time to create 1 rle with old method : 0.07468795776367188 time for calcul the mask position with numpy : 0.05954885482788086 nb_pixel_total : 220064 time to create 1 rle with new method : 0.661672830581665 time for calcul the mask position with numpy : 0.052484989166259766 nb_pixel_total : 142 time to create 1 rle with old method : 0.00034117698669433594 time for calcul the mask position with numpy : 0.052532196044921875 nb_pixel_total : 83311 time to create 1 rle with old method : 0.09078478813171387 time for calcul the mask position with numpy : 0.055428266525268555 nb_pixel_total : 157457 time to create 1 rle with new method : 1.4592573642730713 time for calcul the mask position with numpy : 0.053151845932006836 nb_pixel_total : 49629 time to create 1 rle with old method : 0.06872439384460449 time for calcul the mask position with numpy : 0.055017948150634766 nb_pixel_total : 206600 time to create 1 rle with new method : 1.429138422012329 time for calcul the mask position with numpy : 0.05385255813598633 nb_pixel_total : 105461 time to create 1 rle with old method : 0.11284375190734863 time for calcul the mask position with numpy : 0.051793813705444336 nb_pixel_total : 98142 time to create 1 rle with old method : 0.1059422492980957 time for calcul the mask position with numpy : 0.05259060859680176 nb_pixel_total : 28023 time to create 1 rle with old method : 0.029571533203125 time for calcul the mask position with numpy : 0.05348992347717285 nb_pixel_total : 40831 time to create 1 rle with old method : 0.04297518730163574 time for calcul the mask position with numpy : 0.05172133445739746 nb_pixel_total : 73214 time to create 1 rle with old method : 0.07773160934448242 time for calcul the mask position with numpy : 0.05276322364807129 nb_pixel_total : 30094 time to create 1 rle with old method : 0.03213667869567871 time for calcul the mask position with numpy : 0.05127382278442383 nb_pixel_total : 44410 time to create 1 rle with old method : 0.0485076904296875 time for calcul the mask position with numpy : 0.05223727226257324 nb_pixel_total : 164919 time to create 1 rle with new method : 1.2912547588348389 time for calcul the mask position with numpy : 0.05494427680969238 nb_pixel_total : 102119 time to create 1 rle with old method : 0.11480259895324707 time for calcul the mask position with numpy : 0.05622267723083496 nb_pixel_total : 140110 time to create 1 rle with old method : 0.1586446762084961 time for calcul the mask position with numpy : 0.05359959602355957 nb_pixel_total : 49507 time to create 1 rle with old method : 0.05544447898864746 time for calcul the mask position with numpy : 0.053043365478515625 nb_pixel_total : 50734 time to create 1 rle with old method : 0.055977582931518555 time for calcul the mask position with numpy : 0.05397510528564453 nb_pixel_total : 115768 time to create 1 rle with old method : 0.12600469589233398 time for calcul the mask position with numpy : 0.05307435989379883 nb_pixel_total : 141018 time to create 1 rle with old method : 0.15218615531921387 time for calcul the mask position with numpy : 0.0505063533782959 nb_pixel_total : 1343 time to create 1 rle with old method : 0.0018107891082763672 time for calcul the mask position with numpy : 0.05302143096923828 nb_pixel_total : 99594 time to create 1 rle with old method : 0.10618472099304199 time for calcul the mask position with numpy : 0.07233834266662598 nb_pixel_total : 559176 time to create 1 rle with new method : 1.6055502891540527 time for calcul the mask position with numpy : 0.054776906967163086 nb_pixel_total : 210655 time to create 1 rle with new method : 1.7329230308532715 time for calcul the mask position with numpy : 0.0576021671295166 nb_pixel_total : 65689 time to create 1 rle with old method : 0.07602906227111816 time for calcul the mask position with numpy : 0.05238485336303711 nb_pixel_total : 58698 time to create 1 rle with old method : 0.06612992286682129 time for calcul the mask position with numpy : 0.05170297622680664 nb_pixel_total : 20957 time to create 1 rle with old method : 0.023540258407592773 time for calcul the mask position with numpy : 0.06029796600341797 nb_pixel_total : 597591 time to create 1 rle with new method : 1.4821968078613281 time for calcul the mask position with numpy : 0.05243659019470215 nb_pixel_total : 5855 time to create 1 rle with old method : 0.006644725799560547 time for calcul the mask position with numpy : 0.05162978172302246 nb_pixel_total : 112450 time to create 1 rle with old method : 0.11984705924987793 time for calcul the mask position with numpy : 0.05309152603149414 nb_pixel_total : 226197 time to create 1 rle with new method : 1.2179028987884521 time for calcul the mask position with numpy : 0.051629066467285156 nb_pixel_total : 1041 time to create 1 rle with old method : 0.0012323856353759766 time for calcul the mask position with numpy : 0.05313587188720703 nb_pixel_total : 53999 time to create 1 rle with old method : 0.05865216255187988 time for calcul the mask position with numpy : 0.05608201026916504 nb_pixel_total : 438321 time to create 1 rle with new method : 1.2357280254364014 time for calcul the mask position with numpy : 0.05324101448059082 nb_pixel_total : 38678 time to create 1 rle with old method : 0.04250812530517578 time for calcul the mask position with numpy : 0.0597081184387207 nb_pixel_total : 637579 time to create 1 rle with new method : 1.2618591785430908 time for calcul the mask position with numpy : 0.0530085563659668 nb_pixel_total : 86527 time to create 1 rle with old method : 0.09770894050598145 time for calcul the mask position with numpy : 0.05649232864379883 nb_pixel_total : 5300 time to create 1 rle with old method : 0.006526947021484375 time for calcul the mask position with numpy : 0.055170297622680664 nb_pixel_total : 126443 time to create 1 rle with old method : 0.13477730751037598 time for calcul the mask position with numpy : 0.05457162857055664 nb_pixel_total : 134334 time to create 1 rle with old method : 0.15318608283996582 time for calcul the mask position with numpy : 0.05549883842468262 nb_pixel_total : 187397 time to create 1 rle with new method : 1.2439029216766357 time for calcul the mask position with numpy : 0.05171322822570801 nb_pixel_total : 13895 time to create 1 rle with old method : 0.015145063400268555 time for calcul the mask position with numpy : 0.056700944900512695 nb_pixel_total : 54668 time to create 1 rle with old method : 0.07786989212036133 time for calcul the mask position with numpy : 0.0576329231262207 nb_pixel_total : 490030 time to create 1 rle with new method : 1.4263033866882324 time for calcul the mask position with numpy : 0.05576920509338379 nb_pixel_total : 43185 time to create 1 rle with old method : 0.05002331733703613 time for calcul the mask position with numpy : 0.06668901443481445 nb_pixel_total : 398531 time to create 1 rle with new method : 1.2806930541992188 time for calcul the mask position with numpy : 0.05576801300048828 nb_pixel_total : 8245 time to create 1 rle with old method : 0.010193109512329102 time for calcul the mask position with numpy : 0.06035280227661133 nb_pixel_total : 72234 time to create 1 rle with old method : 0.08542680740356445 time for calcul the mask position with numpy : 0.05471372604370117 nb_pixel_total : 8007 time to create 1 rle with old method : 0.009768247604370117 create new chi : 33.1067681312561 time to delete rle : 0.009521722793579102 batch 1 Loaded 71 chid ids of type : 4855 Number RLEs to save : 52563 TO DO : save crop sub photo not yet done ! save time : 8.391074180603027 nb_obj : 8 nb_hashtags : 3 time to prepare the origin masks : 27.89553213119507 time for calcul the mask position with numpy : 0.6522948741912842 nb_pixel_total : 3718961 time to create 1 rle with new method : 1.9214568138122559 time for calcul the mask position with numpy : 0.06948447227478027 nb_pixel_total : 41923 time to create 1 rle with old method : 0.047472238540649414 time for calcul the mask position with numpy : 0.07079267501831055 nb_pixel_total : 289324 time to create 1 rle with new method : 1.3151090145111084 time for calcul the mask position with numpy : 0.07046961784362793 nb_pixel_total : 57813 time to create 1 rle with old method : 0.0632624626159668 time for calcul the mask position with numpy : 0.06183791160583496 nb_pixel_total : 31273 time to create 1 rle with old method : 0.034050941467285156 time for calcul the mask position with numpy : 0.0710606575012207 nb_pixel_total : 206726 time to create 1 rle with new method : 1.3461754322052002 time for calcul the mask position with numpy : 0.1299128532409668 nb_pixel_total : 774020 time to create 1 rle with new method : 1.3922958374023438 time for calcul the mask position with numpy : 0.06490087509155273 nb_pixel_total : 99427 time to create 1 rle with old method : 0.10742878913879395 time for calcul the mask position with numpy : 0.45160555839538574 nb_pixel_total : 7759733 time to create 1 rle with new method : 1.4683701992034912 create new chi : 9.641964435577393 time to delete rle : 0.002486705780029297 batch 1 Loaded 9 chid ids of type : 4855 Number RLEs to save : 20095 TO DO : save crop sub photo not yet done ! save time : 2.6953089237213135 map_output_result : {1379844701: (0.5184859099901696, 'Should be the crop_list due to order', 0.14860394291832932), 1379844407: (0.5184859099901696, 'Should be the crop_list due to order', 0.40685378705217984), 1379844391: (0.5184859099901696, 'Should be the crop_list due to order', 1.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1379844701, 1379844407, 1379844391] Looping around the photos to save general results len do output : 3 /1379844701.Didn't retrieve data . /1379844407.Didn't retrieve data . /1379844391.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '3623726') ('4741', '26290978', '1379844701', None, None, None, None, None, '3623726') ('4741', None, None, None, None, None, None, None, '3623726') ('4741', '26290978', '1379844407', None, None, None, None, None, '3623726') ('4741', None, None, None, None, None, None, None, '3623726') ('4741', '26290978', '1379844391', None, None, None, None, None, '3623726') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.017011642456054688 save_final save missing photos in datou_result : time spend for datou_step_exec : 276.27536392211914 time spend to save output : 0.018793821334838867 total time spend for step 5 : 276.294157743454 step6:crop_condition Thu Aug 28 11:48:16 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4855 Loading chi in step crop for list_pids : 3 ! batch 1 Loaded 149 chid ids of type : 4855 begin to crop the class : papier param for this class : {'min_score': 0.1} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 22 About to insert : list_path_to_insert length 22 new photo from crops ! About to upload 22 photos upload in portfolio : 3015255 init cache_photo without model_param we have 22 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756374528_3404830 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 22 photos in the portfolio 3015255 time of upload the photos Elapsed time : 5.945039987564087 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.1} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 33 About to insert : list_path_to_insert length 33 new photo from crops ! About to upload 33 photos upload in portfolio : 3015255 init cache_photo without model_param we have 33 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756374555_3404830 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 33 photos in the portfolio 3015255 time of upload the photos Elapsed time : 9.128208637237549 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.1} filtre for class : metal hashtag_id of this class : 492628673 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 3015255 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756374567_3404830 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 4 photos in the portfolio 3015255 time of upload the photos Elapsed time : 1.6093072891235352 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.1} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! About to upload 8 photos upload in portfolio : 3015255 init cache_photo without model_param we have 8 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756374573_3404830 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 8 photos in the portfolio 3015255 time of upload the photos Elapsed time : 2.5373990535736084 we have finished the crop for the class : pet_clair begin to crop the class : pehd param for this class : {'min_score': 0.1} filtre for class : pehd hashtag_id of this class : 628944319 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3015255 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756374577_3404830 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 1 photos in the portfolio 3015255 time of upload the photos Elapsed time : 0.5989058017730713 we have finished the crop for the class : pehd begin to crop the class : pet_fonce param for this class : {'min_score': 0.1} filtre for class : pet_fonce hashtag_id of this class : 2107755900 begin to crop the class : sac param for this class : {'min_score': 0.1} filtre for class : sac hashtag_id of this class : 492618350 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! About to upload 7 photos upload in portfolio : 3015255 init cache_photo without model_param we have 7 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756374584_3404830 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 7 photos in the portfolio 3015255 time of upload the photos Elapsed time : 1.872105360031128 we have finished the crop for the class : sac begin to crop the class : textiles param for this class : {'min_score': 0.1} filtre for class : textiles hashtag_id of this class : 494789423 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3015255 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756374588_3404830 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 1 photos in the portfolio 3015255 time of upload the photos Elapsed time : 0.6308012008666992 we have finished the crop for the class : textiles begin to crop the class : verre param for this class : {'min_score': 0.1} filtre for class : verre hashtag_id of this class : 492631919 begin to crop the class : organique param for this class : {'min_score': 0.1} filtre for class : organique hashtag_id of this class : 530726390 begin to crop the class : dasri param for this class : {'min_score': 0.1} filtre for class : dasri hashtag_id of this class : 1608424396 begin to crop the class : masque param for this class : {'min_score': 0.1} filtre for class : masque hashtag_id of this class : 492626781 begin to crop the class : encombrant param for this class : {'min_score': 0.1} filtre for class : encombrant hashtag_id of this class : 605975549 begin to crop the class : autre param for this class : {'min_score': 0.1} filtre for class : autre hashtag_id of this class : 494826614 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 14 About to insert : list_path_to_insert length 14 new photo from crops ! About to upload 14 photos upload in portfolio : 3015255 init cache_photo without model_param we have 14 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756374601_3404830 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 14 photos in the portfolio 3015255 time of upload the photos Elapsed time : 3.8888471126556396 we have finished the crop for the class : autre begin to crop the class : autre_emballage param for this class : {'min_score': 0.1} filtre for class : autre_emballage hashtag_id of this class : 2107760328 begin to crop the class : autre_non_emballage param for this class : {'min_score': 0.1} filtre for class : autre_non_emballage hashtag_id of this class : 2107760329 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 3015255 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756374608_3404830 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 5 photos in the portfolio 3015255 time of upload the photos Elapsed time : 1.394049882888794 we have finished the crop for the class : autre_non_emballage begin to crop the class : environnement param for this class : {'min_score': 0.1} filtre for class : environnement hashtag_id of this class : 493012381 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 12 About to insert : list_path_to_insert length 12 new photo from crops ! About to upload 12 photos upload in portfolio : 3015255 init cache_photo without model_param we have 12 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756374668_3404830 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 12 photos in the portfolio 3015255 time of upload the photos Elapsed time : 3.9258179664611816 we have finished the crop for the class : environnement delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1379844701, 1379844407, 1379844391] Looping around the photos to save general results len do output : 107 /1379850859Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850860Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850862Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850864Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850865Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850867Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850868Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850869Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850871Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850872Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850873Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850875Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850876Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850877Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850879Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850880Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850881Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850883Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850884Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850885Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850886Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850888Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850955Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850956Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850958Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850960Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850961Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850962Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850963Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850964Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850965Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850966Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850967Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850968Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850969Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850970Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850971Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850972Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850973Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850974Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850975Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850976Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850977Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850978Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850979Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850980Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850981Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850982Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850983Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850984Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850985Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850986Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850987Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850988Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850989Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850994Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850995Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850996Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379850997Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851025Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851026Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851027Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851028Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851029Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851030Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851031Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851032Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851034Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851037Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851038Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851039Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851040Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851041Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851042Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851043Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851044Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851047Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851048Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851049Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851050Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851051Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851052Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851053Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851054Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851055Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851056Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851057Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851058Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851059Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851060Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851062Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851063Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851064Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851065Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851066Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851340Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851341Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851343Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851344Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851345Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851347Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851348Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851349Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851351Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851352Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851354Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379851355Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '3623726') ('4741', '26290978', '1379844701', None, None, None, None, None, '3623726') ('4741', None, None, None, None, None, None, None, '3623726') ('4741', '26290978', '1379844407', None, None, None, None, None, '3623726') ('4741', None, None, None, None, None, None, None, '3623726') ('4741', '26290978', '1379844391', None, None, None, None, None, '3623726') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 324 time used for this insertion : 0.05899310111999512 save_final save missing photos in datou_result : time spend for datou_step_exec : 178.24579286575317 time spend to save output : 0.06573772430419922 total time spend for step 6 : 178.31153059005737 step7:ventilate_hashtags_in_portfolio Thu Aug 28 11:51:15 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 26290978 get user id for portfolio 26290978 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26290978 AND mptpi.`type`=4855 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26290978 AND mptpi.`type`=4855 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26290978 AND mptpi.`type`=4856 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26290978 AND mptpi.`type`=4856 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do lien utilise dans velours : https://www.fotonower.com/velours/26291235,26291236,26291237,26291238,26291239,26291244,26291246,26291247,26291248,26291249,26291250,26291251,26291253,26291254,26291255,26291256,26291257,26291258,26291259,26291260?tags=papier,carton,metal,pet_clair,pehd,ela,textiles,verre,organique,dasri,masque,encombrant,autre_non_emballage,environnement,autre,flux_dev,mal_croppe,flou,film_dev_souple,sac_om_plein&datou_id_consolidate=4742&port_consolidate=26290978 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1379844701, 1379844407, 1379844391] Looping around the photos to save general results len do output : 1 /26290978. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '3623726') ('4741', '26290978', '1379844701', None, None, None, None, None, '3623726') ('4741', None, None, None, None, None, None, None, '3623726') ('4741', '26290978', '1379844407', None, None, None, None, None, '3623726') ('4741', None, None, None, None, None, None, None, '3623726') ('4741', '26290978', '1379844391', None, None, None, None, None, '3623726') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 4 time used for this insertion : 0.016482114791870117 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.67070746421814 time spend to save output : 0.016687631607055664 total time spend for step 7 : 4.687395095825195 step8:final Thu Aug 28 11:51:19 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1379844701: ('0.28105211116339807',), 1379844407: ('0.28105211116339807',), 1379844391: ('0.28105211116339807',)} new output for save of step final : {1379844701: ('0.28105211116339807',), 1379844407: ('0.28105211116339807',), 1379844391: ('0.28105211116339807',)} [1379844701, 1379844407, 1379844391] Looping around the photos to save general results len do output : 3 /1379844701.Didn't retrieve data . /1379844407.Didn't retrieve data . /1379844391.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '3623726') ('4741', '26290978', '1379844701', None, None, None, None, None, '3623726') ('4741', None, None, None, None, None, None, None, '3623726') ('4741', '26290978', '1379844407', None, None, None, None, None, '3623726') ('4741', None, None, None, None, None, None, None, '3623726') ('4741', '26290978', '1379844391', None, None, None, None, None, '3623726') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.01536417007446289 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.11939072608947754 time spend to save output : 0.015639066696166992 total time spend for step 8 : 0.13502979278564453 step9:velours_tree Thu Aug 28 11:51:20 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.26230692863464355 time spend to save output : 3.910064697265625e-05 total time spend for step 9 : 0.2623460292816162 step10:send_mail_cod Thu Aug 28 11:51:20 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P26290978_28-08-2025_11_51_20.pdf 26291209 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette262912091756374680 26291210 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette262912101756374682 26291211 change filename to text .change filename to text .change filename to text .change filename to text .imagette262912111756374683 26291212 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette262912121756374684 26291213 change filename to text .imagette262912131756374684 26291214 imagette262912141756374684 26291215 imagette262912151756374684 26291216 imagette262912161756374684 26291217 imagette262912171756374684 26291218 imagette262912181756374684 26291219 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette262912191756374684 26291220 change filename to text .imagette262912201756374685 26291221 imagette262912211756374685 26291222 imagette262912221756374685 26291223 imagette262912231756374685 26291224 imagette262912241756374685 26291225 imagette262912251756374685 26291226 imagette262912261756374685 26291227 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette262912271756374685 26291229 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette262912291756374686 26291230 imagette262912301756374687 26291231 imagette262912311756374687 26291232 imagette262912321756374687 26291233 imagette262912331756374687 26291234 imagette262912341756374687 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=26290978 and hashtag_type = 4855 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/26291235,26291236,26291237,26291238,26291239,26291244,26291246,26291247,26291248,26291249,26291250,26291251,26291253,26291254,26291255,26291256,26291257,26291258,26291259,26291260?tags=papier,carton,metal,pet_clair,pehd,ela,textiles,verre,organique,dasri,masque,encombrant,autre_non_emballage,environnement,autre,flux_dev,mal_croppe,flou,film_dev_souple,sac_om_plein&datou_id_consolidate=4742&port_consolidate=26290978 your option no_mail is active, we will not send the real mail to your client args[1379844701] : ((1379844701, 991.1598989500137, 2107751945), (1379844701, -0.3321167275213726, 496442774), '0.28105211116339807') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1379844407] : ((1379844407, 1172.3631311147594, 2107751945), (1379844407, -0.15899188593756994, 496442774), '0.28105211116339807') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1379844391] : ((1379844391, 276.4735240769179, 492688767), (1379844391, -0.640083973195245, 501862349), '0.28105211116339807') We are sending mail with results at report@fotonower.com,cod@fotonower.com refus_total : 0.28105211116339807 2022-04-13 10:29:59 0 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26290978_28-08-2025_11_51_20.pdf results_Auto_P26290978_28-08-2025_11_51_20.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26290978_28-08-2025_11_51_20.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4741','26290978','results_Auto_P26290978_28-08-2025_11_51_20.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26290978_28-08-2025_11_51_20.pdf','pdf','','1.94','0.28105211116339807') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1379844701, 1379844407, 1379844391] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '3623726') ('4741', '26290978', '1379844701', None, None, None, None, None, '3623726') ('4741', None, None, None, None, None, None, None, '3623726') ('4741', '26290978', '1379844407', None, None, None, None, None, '3623726') ('4741', None, None, None, None, None, None, None, '3623726') ('4741', '26290978', '1379844391', None, None, None, None, None, '3623726') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.018805503845214844 save_final save missing photos in datou_result : time spend for datou_step_exec : 10.115416765213013 time spend to save output : 0.018955469131469727 total time spend for step 10 : 10.134372234344482 step11:split_time_score Thu Aug 28 11:51:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('44', 3),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 22082025 26290978 Nombre de photos uploadées : 3 / 23040 (0%) 22082025 26290978 Nombre de photos taguées (types de déchets): 0 / 3 (0%) 22082025 26290978 Nombre de photos taguées (volume) : 0 / 3 (0%) elapsed_time : load_data_split_time_score 1.6689300537109375e-06 elapsed_time : order_list_meta_photo_and_scores 5.245208740234375e-06 ??? elapsed_time : fill_and_build_computed_from_old_data 0.00026106834411621094 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.2042863368988037 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26290977 order by id desc limit 1 Qualite : 0.28105211116339807 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26290978_28-08-2025_11_51_20.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26290978 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26290978 AND mptpi.`type`=4855 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26290980 order by id desc limit 1 NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'22082025': {'nb_upload': 3, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1379844701, 1379844407, 1379844391] Looping around the photos to save general results len do output : 1 /26290978Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '3623726') ('4741', '26290978', '1379844701', None, None, None, None, None, '3623726') ('4741', None, None, None, None, None, None, None, '3623726') ('4741', '26290978', '1379844407', None, None, None, None, None, '3623726') ('4741', None, None, None, None, None, None, None, '3623726') ('4741', '26290978', '1379844391', None, None, None, None, None, '3623726') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 4 time used for this insertion : 0.03515195846557617 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.362851619720459 time spend to save output : 0.03534388542175293 total time spend for step 11 : 2.398195505142212 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 1 len(input) + len(total_photo_id_missing) : 3 set_done_treatment 314.39user 431.25system 10:07.84elapsed 122%CPU (0avgtext+0avgdata 5422664maxresident)k 1286200inputs+364352outputs (14652major+31117746minor)pagefaults 0swaps