python /home/admin/mtr/script_for_cron.py -j default -m 20 -a 'python3 ~/workarea/git/Velours/python/prod/datou.py -j batch_current -a 4433' -s traitement_sts -M 0 -S 0 -U 100,80,95 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/home/admin/workarea/git/apy', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 66244 load datou : 4433 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4433, datou_cur_ids : ['3772637'] with mtr_portfolio_ids : ['27193433'] and first list_photo_ids : [] new path : /proc/66244/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, brightness, blur_detection, crop_condition, thcl, tfhub_classification2, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 15 list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 15 ; length of list_pids : 15 ; length of list_args : 15 time to download the photos : 4.986512899398804 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 14 step1:mask_detect Fri Sep 26 11:27:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 6800 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-09-26 11:27:36.122639: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-26 11:27:36.160559: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-26 11:27:36.162595: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fccdc000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-26 11:27:36.162665: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-26 11:27:36.167004: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-26 11:27:36.330970: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x3063faf0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-26 11:27:36.331033: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-26 11:27:36.332309: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:27:36.334634: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:27:36.346602: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:27:36.357838: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:27:36.359782: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:27:36.369763: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:27:36.373201: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:27:36.396932: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:27:36.398536: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:27:36.398642: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:27:36.400001: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-26 11:27:36.400022: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-26 11:27:36.400034: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-26 11:27:36.402650: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6249 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-09-26 11:27:36.892682: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:27:36.892807: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:27:36.892826: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:27:36.892842: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:27:36.892857: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:27:36.892872: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:27:36.892887: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:27:36.892904: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:27:36.893918: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:27:36.895046: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:27:36.895082: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:27:36.895098: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:27:36.895113: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:27:36.895127: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:27:36.895141: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:27:36.895157: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:27:36.895173: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:27:36.896192: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:27:36.896244: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-26 11:27:36.896254: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-26 11:27:36.896261: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-26 11:27:36.897275: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6249 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-09-26 11:27:53.512160: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:27:53.788546: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:27:56.014133: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.015113: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.60G (3865470464 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.015999: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.24G (3478923264 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.016890: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3131030784 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.017759: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.62G (2817927680 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.018622: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.36G (2536134912 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.019487: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.12G (2282521344 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.019542: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:27:56.021080: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.021116: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:27:56.028546: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.028589: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:27:56.029105: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.029122: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:27:56.037253: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.037304: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 466.56MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:27:56.037825: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.037842: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 466.56MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:27:56.069168: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.069221: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:27:56.069749: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.069766: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:27:56.078363: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.078480: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 243.25MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:27:56.079480: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.079511: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 243.25MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:27:56.115083: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.115643: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.120015: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.120542: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.163009: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.163573: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.165476: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.166000: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.197753: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.198320: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.199841: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.200376: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.212705: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.213306: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.214952: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.215471: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.226230: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.226775: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.240578: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.241112: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.428301: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.428856: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.429374: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.429867: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.444789: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.445311: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.492680: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.493213: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.493728: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.494240: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.522928: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.523492: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.524013: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.528935: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.554127: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.554663: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.573308: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.573835: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.595370: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.595901: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.603433: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.603984: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.604518: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.605038: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.605729: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.606255: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.630956: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.631504: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.632039: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.632559: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.633091: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.633618: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.634130: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.634639: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.654248: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.654774: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.686761: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.687316: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.784522: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.785118: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.785682: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.786215: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.807768: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.808356: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.848218: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.848918: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.849565: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.850200: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.856796: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.857319: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.857853: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.858375: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.860868: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.860892: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-09-26 11:27:56.872985: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.873513: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.899726: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.900259: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.900816: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.901354: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.901902: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:27:56.902431: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 15 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 13 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 15 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 11 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 1.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 4.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 1 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 2 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 2 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 1 Detection mask done ! Trying to reset tf kernel 66516 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 9 tf kernel not reseted sub process len(results) : 15 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 15 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 6800 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.0008082389831542969 nb_pixel_total : 19900 time to create 1 rle with old method : 0.027335166931152344 length of segment : 182 time for calcul the mask position with numpy : 0.004013776779174805 nb_pixel_total : 148594 time to create 1 rle with old method : 0.17198848724365234 length of segment : 449 time for calcul the mask position with numpy : 0.0014083385467529297 nb_pixel_total : 90742 time to create 1 rle with old method : 0.09837126731872559 length of segment : 413 time for calcul the mask position with numpy : 0.0011703968048095703 nb_pixel_total : 93220 time to create 1 rle with old method : 0.09948468208312988 length of segment : 272 time for calcul the mask position with numpy : 0.0017313957214355469 nb_pixel_total : 119993 time to create 1 rle with old method : 0.13025808334350586 length of segment : 309 time for calcul the mask position with numpy : 0.0010597705841064453 nb_pixel_total : 49385 time to create 1 rle with old method : 0.05310964584350586 length of segment : 439 time for calcul the mask position with numpy : 0.0013759136199951172 nb_pixel_total : 32100 time to create 1 rle with old method : 0.03511381149291992 length of segment : 397 time for calcul the mask position with numpy : 0.029830217361450195 nb_pixel_total : 1034057 time to create 1 rle with new method : 0.4504513740539551 length of segment : 2638 time for calcul the mask position with numpy : 0.00654292106628418 nb_pixel_total : 351100 time to create 1 rle with new method : 0.024353981018066406 length of segment : 1288 time for calcul the mask position with numpy : 0.0026559829711914062 nb_pixel_total : 97833 time to create 1 rle with old method : 0.10500526428222656 length of segment : 435 time for calcul the mask position with numpy : 0.0022351741790771484 nb_pixel_total : 65683 time to create 1 rle with old method : 0.07341670989990234 length of segment : 346 time for calcul the mask position with numpy : 0.010471105575561523 nb_pixel_total : 432368 time to create 1 rle with new method : 0.022487640380859375 length of segment : 1077 time for calcul the mask position with numpy : 0.00109100341796875 nb_pixel_total : 37644 time to create 1 rle with old method : 0.0406646728515625 length of segment : 194 time for calcul the mask position with numpy : 0.00037169456481933594 nb_pixel_total : 6462 time to create 1 rle with old method : 0.007410764694213867 length of segment : 162 time for calcul the mask position with numpy : 0.0012099742889404297 nb_pixel_total : 39328 time to create 1 rle with old method : 0.04558062553405762 length of segment : 368 time for calcul the mask position with numpy : 0.0029184818267822266 nb_pixel_total : 121773 time to create 1 rle with old method : 0.1370093822479248 length of segment : 711 time for calcul the mask position with numpy : 0.0027227401733398438 nb_pixel_total : 139644 time to create 1 rle with old method : 0.15262699127197266 length of segment : 529 time for calcul the mask position with numpy : 0.030347824096679688 nb_pixel_total : 1207576 time to create 1 rle with new method : 0.23276185989379883 length of segment : 3059 time for calcul the mask position with numpy : 0.0029265880584716797 nb_pixel_total : 135603 time to create 1 rle with old method : 0.14997386932373047 length of segment : 597 time for calcul the mask position with numpy : 0.0030698776245117188 nb_pixel_total : 122145 time to create 1 rle with old method : 0.13762140274047852 length of segment : 524 time for calcul the mask position with numpy : 0.0005474090576171875 nb_pixel_total : 11252 time to create 1 rle with old method : 0.01261281967163086 length of segment : 139 time for calcul the mask position with numpy : 0.001088857650756836 nb_pixel_total : 34102 time to create 1 rle with old method : 0.03863358497619629 length of segment : 195 time for calcul the mask position with numpy : 0.0012068748474121094 nb_pixel_total : 54286 time to create 1 rle with old method : 0.061104774475097656 length of segment : 162 time for calcul the mask position with numpy : 0.0009412765502929688 nb_pixel_total : 36510 time to create 1 rle with old method : 0.03930783271789551 length of segment : 116 time for calcul the mask position with numpy : 0.003553152084350586 nb_pixel_total : 121925 time to create 1 rle with old method : 0.13206267356872559 length of segment : 511 time for calcul the mask position with numpy : 0.0026319026947021484 nb_pixel_total : 111567 time to create 1 rle with old method : 0.11858272552490234 length of segment : 366 time for calcul the mask position with numpy : 0.041385650634765625 nb_pixel_total : 1200652 time to create 1 rle with new method : 0.3266172409057617 length of segment : 2301 time for calcul the mask position with numpy : 0.00947117805480957 nb_pixel_total : 362535 time to create 1 rle with new method : 0.012821674346923828 length of segment : 1196 time for calcul the mask position with numpy : 0.0007765293121337891 nb_pixel_total : 50789 time to create 1 rle with old method : 0.05979347229003906 length of segment : 149 time for calcul the mask position with numpy : 0.0041773319244384766 nb_pixel_total : 160247 time to create 1 rle with new method : 0.006200313568115234 length of segment : 457 time for calcul the mask position with numpy : 0.0042896270751953125 nb_pixel_total : 133103 time to create 1 rle with old method : 0.1628129482269287 length of segment : 507 time for calcul the mask position with numpy : 0.00045228004455566406 nb_pixel_total : 16474 time to create 1 rle with old method : 0.0243377685546875 length of segment : 197 time for calcul the mask position with numpy : 0.0027658939361572266 nb_pixel_total : 80098 time to create 1 rle with old method : 0.10465574264526367 length of segment : 355 time for calcul the mask position with numpy : 0.0007336139678955078 nb_pixel_total : 17470 time to create 1 rle with old method : 0.020339012145996094 length of segment : 286 time for calcul the mask position with numpy : 0.024615764617919922 nb_pixel_total : 691791 time to create 1 rle with new method : 0.21816682815551758 length of segment : 2340 time for calcul the mask position with numpy : 0.0013737678527832031 nb_pixel_total : 74250 time to create 1 rle with old method : 0.08664774894714355 length of segment : 348 time for calcul the mask position with numpy : 0.0014510154724121094 nb_pixel_total : 39182 time to create 1 rle with old method : 0.0449528694152832 length of segment : 351 time for calcul the mask position with numpy : 0.0005917549133300781 nb_pixel_total : 30719 time to create 1 rle with old method : 0.03581547737121582 length of segment : 145 time for calcul the mask position with numpy : 0.0005385875701904297 nb_pixel_total : 18613 time to create 1 rle with old method : 0.022019624710083008 length of segment : 87 time for calcul the mask position with numpy : 0.004743337631225586 nb_pixel_total : 220890 time to create 1 rle with new method : 0.012958765029907227 length of segment : 873 time for calcul the mask position with numpy : 0.0051615238189697266 nb_pixel_total : 224670 time to create 1 rle with new method : 0.009396553039550781 length of segment : 449 time for calcul the mask position with numpy : 0.0007510185241699219 nb_pixel_total : 27945 time to create 1 rle with old method : 0.03215599060058594 length of segment : 125 time for calcul the mask position with numpy : 0.004239559173583984 nb_pixel_total : 198621 time to create 1 rle with new method : 0.007180929183959961 length of segment : 694 time for calcul the mask position with numpy : 0.008249044418334961 nb_pixel_total : 381995 time to create 1 rle with new method : 0.014611005783081055 length of segment : 1014 time for calcul the mask position with numpy : 0.003994941711425781 nb_pixel_total : 150164 time to create 1 rle with new method : 0.00681614875793457 length of segment : 473 time for calcul the mask position with numpy : 0.03097701072692871 nb_pixel_total : 905105 time to create 1 rle with new method : 0.396404504776001 length of segment : 2722 time for calcul the mask position with numpy : 0.001033782958984375 nb_pixel_total : 18379 time to create 1 rle with old method : 0.021009445190429688 length of segment : 261 time for calcul the mask position with numpy : 0.0009450912475585938 nb_pixel_total : 48555 time to create 1 rle with old method : 0.055703163146972656 length of segment : 252 time for calcul the mask position with numpy : 0.0009295940399169922 nb_pixel_total : 35824 time to create 1 rle with old method : 0.03983592987060547 length of segment : 427 time for calcul the mask position with numpy : 0.001714468002319336 nb_pixel_total : 144626 time to create 1 rle with old method : 0.1583387851715088 length of segment : 442 time for calcul the mask position with numpy : 0.008069515228271484 nb_pixel_total : 412344 time to create 1 rle with new method : 0.012471914291381836 length of segment : 770 time for calcul the mask position with numpy : 0.002901315689086914 nb_pixel_total : 89881 time to create 1 rle with old method : 0.10502338409423828 length of segment : 587 time for calcul the mask position with numpy : 0.0008013248443603516 nb_pixel_total : 30655 time to create 1 rle with old method : 0.034540653228759766 length of segment : 168 time for calcul the mask position with numpy : 0.007525205612182617 nb_pixel_total : 377502 time to create 1 rle with new method : 0.014545440673828125 length of segment : 955 time for calcul the mask position with numpy : 0.04023122787475586 nb_pixel_total : 851091 time to create 1 rle with new method : 0.26679348945617676 length of segment : 2359 time for calcul the mask position with numpy : 0.005396127700805664 nb_pixel_total : 266356 time to create 1 rle with new method : 0.008475542068481445 length of segment : 911 time for calcul the mask position with numpy : 0.0007410049438476562 nb_pixel_total : 33472 time to create 1 rle with old method : 0.0387113094329834 length of segment : 206 time for calcul the mask position with numpy : 0.0059053897857666016 nb_pixel_total : 352228 time to create 1 rle with new method : 0.011939525604248047 length of segment : 1102 time for calcul the mask position with numpy : 0.0001239776611328125 nb_pixel_total : 4276 time to create 1 rle with old method : 0.004978656768798828 length of segment : 118 time for calcul the mask position with numpy : 0.0003361701965332031 nb_pixel_total : 15628 time to create 1 rle with old method : 0.01743912696838379 length of segment : 176 time for calcul the mask position with numpy : 0.0049343109130859375 nb_pixel_total : 296831 time to create 1 rle with new method : 0.008250713348388672 length of segment : 669 time for calcul the mask position with numpy : 0.0005979537963867188 nb_pixel_total : 21168 time to create 1 rle with old method : 0.023503541946411133 length of segment : 189 time for calcul the mask position with numpy : 0.005019187927246094 nb_pixel_total : 288367 time to create 1 rle with new method : 0.00890040397644043 length of segment : 794 time for calcul the mask position with numpy : 0.0032393932342529297 nb_pixel_total : 234919 time to create 1 rle with new method : 0.009896278381347656 length of segment : 888 time for calcul the mask position with numpy : 0.0011246204376220703 nb_pixel_total : 68013 time to create 1 rle with old method : 0.08324527740478516 length of segment : 214 time for calcul the mask position with numpy : 0.0004074573516845703 nb_pixel_total : 8967 time to create 1 rle with old method : 0.010843753814697266 length of segment : 122 time for calcul the mask position with numpy : 0.011645078659057617 nb_pixel_total : 460111 time to create 1 rle with new method : 0.01999831199645996 length of segment : 1161 time for calcul the mask position with numpy : 0.020276308059692383 nb_pixel_total : 528111 time to create 1 rle with new method : 0.06432366371154785 length of segment : 1281 time for calcul the mask position with numpy : 0.0005519390106201172 nb_pixel_total : 7240 time to create 1 rle with old method : 0.008198738098144531 length of segment : 146 time for calcul the mask position with numpy : 0.0010657310485839844 nb_pixel_total : 25632 time to create 1 rle with old method : 0.028184890747070312 length of segment : 269 time for calcul the mask position with numpy : 0.057256460189819336 nb_pixel_total : 1278928 time to create 1 rle with new method : 0.4516632556915283 length of segment : 2063 time for calcul the mask position with numpy : 0.0025360584259033203 nb_pixel_total : 60006 time to create 1 rle with old method : 0.0649569034576416 length of segment : 375 time for calcul the mask position with numpy : 0.00042891502380371094 nb_pixel_total : 12385 time to create 1 rle with old method : 0.013486623764038086 length of segment : 126 time for calcul the mask position with numpy : 0.004314422607421875 nb_pixel_total : 271777 time to create 1 rle with new method : 0.014693021774291992 length of segment : 492 time for calcul the mask position with numpy : 0.00032782554626464844 nb_pixel_total : 6110 time to create 1 rle with old method : 0.007524013519287109 length of segment : 139 time for calcul the mask position with numpy : 0.0012359619140625 nb_pixel_total : 61390 time to create 1 rle with old method : 0.07586336135864258 length of segment : 266 time for calcul the mask position with numpy : 0.0032949447631835938 nb_pixel_total : 129821 time to create 1 rle with old method : 0.14733648300170898 length of segment : 568 time for calcul the mask position with numpy : 0.0047037601470947266 nb_pixel_total : 190706 time to create 1 rle with new method : 0.011897563934326172 length of segment : 550 time for calcul the mask position with numpy : 0.002576589584350586 nb_pixel_total : 81986 time to create 1 rle with old method : 0.09283614158630371 length of segment : 393 time for calcul the mask position with numpy : 0.03534960746765137 nb_pixel_total : 1202920 time to create 1 rle with new method : 0.13013029098510742 length of segment : 3199 time for calcul the mask position with numpy : 0.0027573108673095703 nb_pixel_total : 115744 time to create 1 rle with old method : 0.13530397415161133 length of segment : 397 time for calcul the mask position with numpy : 0.0007534027099609375 nb_pixel_total : 27556 time to create 1 rle with old method : 0.029544353485107422 length of segment : 185 time for calcul the mask position with numpy : 0.0018792152404785156 nb_pixel_total : 66688 time to create 1 rle with old method : 0.07286214828491211 length of segment : 425 time for calcul the mask position with numpy : 0.0010676383972167969 nb_pixel_total : 68103 time to create 1 rle with old method : 0.07621884346008301 length of segment : 393 time for calcul the mask position with numpy : 0.010721921920776367 nb_pixel_total : 419367 time to create 1 rle with new method : 0.024692296981811523 length of segment : 1003 time for calcul the mask position with numpy : 0.005956172943115234 nb_pixel_total : 425934 time to create 1 rle with new method : 0.015198945999145508 length of segment : 1025 time for calcul the mask position with numpy : 0.043050289154052734 nb_pixel_total : 299423 time to create 1 rle with new method : 0.7990882396697998 length of segment : 1631 time for calcul the mask position with numpy : 0.007561683654785156 nb_pixel_total : 265536 time to create 1 rle with new method : 0.0205838680267334 length of segment : 1121 time for calcul the mask position with numpy : 0.00011301040649414062 nb_pixel_total : 973 time to create 1 rle with old method : 0.0012073516845703125 length of segment : 58 time for calcul the mask position with numpy : 0.0033292770385742188 nb_pixel_total : 164393 time to create 1 rle with new method : 0.006262302398681641 length of segment : 435 time for calcul the mask position with numpy : 0.0012896060943603516 nb_pixel_total : 63242 time to create 1 rle with old method : 0.07159614562988281 length of segment : 136 time for calcul the mask position with numpy : 0.030356884002685547 nb_pixel_total : 1344496 time to create 1 rle with new method : 0.526726484298706 length of segment : 2700 time for calcul the mask position with numpy : 0.0007555484771728516 nb_pixel_total : 14308 time to create 1 rle with old method : 0.02434396743774414 length of segment : 167 time for calcul the mask position with numpy : 0.0007050037384033203 nb_pixel_total : 21595 time to create 1 rle with old method : 0.03591585159301758 length of segment : 172 time for calcul the mask position with numpy : 0.023708343505859375 nb_pixel_total : 965190 time to create 1 rle with new method : 0.5031225681304932 length of segment : 2748 time for calcul the mask position with numpy : 0.0014290809631347656 nb_pixel_total : 74271 time to create 1 rle with old method : 0.0813140869140625 length of segment : 265 time for calcul the mask position with numpy : 0.008982658386230469 nb_pixel_total : 513286 time to create 1 rle with new method : 0.013300657272338867 length of segment : 841 time for calcul the mask position with numpy : 0.005068063735961914 nb_pixel_total : 259417 time to create 1 rle with new method : 0.008257627487182617 length of segment : 794 time for calcul the mask position with numpy : 0.02928948402404785 nb_pixel_total : 1307150 time to create 1 rle with new method : 0.25158119201660156 length of segment : 2768 time for calcul the mask position with numpy : 0.3803110122680664 nb_pixel_total : 6382164 time to create 1 rle with new method : 1.2922749519348145 length of segment : 4763 time for calcul the mask position with numpy : 0.5784740447998047 nb_pixel_total : 5505983 time to create 1 rle with new method : 0.9516034126281738 length of segment : 6103 time for calcul the mask position with numpy : 0.0010838508605957031 nb_pixel_total : 68031 time to create 1 rle with old method : 0.0774850845336914 length of segment : 301 time for calcul the mask position with numpy : 1.1521713733673096 nb_pixel_total : 9567641 time to create 1 rle with new method : 0.7690696716308594 length of segment : 3052 time for calcul the mask position with numpy : 0.001168966293334961 nb_pixel_total : 69503 time to create 1 rle with old method : 0.07605767250061035 length of segment : 270 time for calcul the mask position with numpy : 0.9666078090667725 nb_pixel_total : 7565334 time to create 1 rle with new method : 1.5234992504119873 length of segment : 5274 time spent for convertir_results : 39.65618133544922 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 105 chid ids of type : 3663 Number RLEs to save : 90612 save missing photos in datou_result : time spend for datou_step_exec : 145.45668053627014 time spend to save output : 4.9409339427948 total time spend for step 1 : 150.39761447906494 step2:brightness Fri Sep 26 11:30:02 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1758878847_66244_1385913856_9ca0ded2129325c941dde24db09235cd.jpg treat image : temp/1758878847_66244_1385913854_5999739dd5bdd4e97fd96ac1b6fbea46.jpg treat image : temp/1758878847_66244_1385913850_5c29c024123e4053bb68c7de8bed0a8b.jpg treat image : temp/1758878847_66244_1385913845_8ddd20ac8b2e903721f9b368b8b0b097.jpg treat image : temp/1758878847_66244_1385913842_6b30039522930f28a772668cc77bbce2.jpg treat image : temp/1758878847_66244_1385913835_ad181f5309a1f7cd9e27d672f083f663.jpg treat image : temp/1758878847_66244_1385913575_a65ca1ebf8465eec79263ecba4eff51d.jpg treat image : temp/1758878847_66244_1385913570_8eb6b902d3d096045fe7139afe9dd77b.jpg treat image : temp/1758878847_66244_1385913567_8cb6c19bc32f74567ae805dd8556d888.jpg treat image : temp/1758878847_66244_1385913565_b52eedcc066c4b070e1565313f2a95d4.jpg treat image : temp/1758878847_66244_1385913562_4d6d805dee73e40f25d8908cbf09c0a1.jpg treat image : temp/1758878847_66244_1385913560_9d075f196ca7c059b85693e34e2d5523.jpg treat image : temp/1758878847_66244_1385913240_987e326d58d1eec5428700c991f3c138.jpg treat image : temp/1758878847_66244_1385913239_1e3fa5c733ac765194aecedab1d45444.jpg treat image : temp/1758878847_66244_1385913238_e837afd0f573bcabf618ef47fc612300.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 15 time used for this insertion : 0.015578746795654297 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 15 time used for this insertion : 0.015676021575927734 save missing photos in datou_result : time spend for datou_step_exec : 24.774983167648315 time spend to save output : 0.047756195068359375 total time spend for step 2 : 24.822739362716675 step3:blur_detection Fri Sep 26 11:30:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1758878847_66244_1385913856_9ca0ded2129325c941dde24db09235cd.jpg resize: (3040, 4056) 1385913856 1.1415852410922487 treat image : temp/1758878847_66244_1385913854_5999739dd5bdd4e97fd96ac1b6fbea46.jpg resize: (3040, 4056) 1385913854 0.4073508800317015 treat image : temp/1758878847_66244_1385913850_5c29c024123e4053bb68c7de8bed0a8b.jpg resize: (3040, 4056) 1385913850 0.3027355961001093 treat image : temp/1758878847_66244_1385913845_8ddd20ac8b2e903721f9b368b8b0b097.jpg resize: (3040, 4056) 1385913845 -2.6777748397735297 treat image : temp/1758878847_66244_1385913842_6b30039522930f28a772668cc77bbce2.jpg resize: (3040, 4056) 1385913842 0.14245055527483566 treat image : temp/1758878847_66244_1385913835_ad181f5309a1f7cd9e27d672f083f663.jpg resize: (3040, 4056) 1385913835 -2.1434955533116296 treat image : temp/1758878847_66244_1385913575_a65ca1ebf8465eec79263ecba4eff51d.jpg resize: (3040, 4056) 1385913575 0.23151719871967572 treat image : temp/1758878847_66244_1385913570_8eb6b902d3d096045fe7139afe9dd77b.jpg resize: (3040, 4056) 1385913570 -0.048049369560924204 treat image : temp/1758878847_66244_1385913567_8cb6c19bc32f74567ae805dd8556d888.jpg resize: (3040, 4056) 1385913567 0.21383111532912 treat image : temp/1758878847_66244_1385913565_b52eedcc066c4b070e1565313f2a95d4.jpg resize: (3040, 4056) 1385913565 0.28614354318155444 treat image : temp/1758878847_66244_1385913562_4d6d805dee73e40f25d8908cbf09c0a1.jpg resize: (3040, 4056) 1385913562 0.23645520213598398 treat image : temp/1758878847_66244_1385913560_9d075f196ca7c059b85693e34e2d5523.jpg resize: (3040, 4056) 1385913560 -0.14602185656991726 treat image : temp/1758878847_66244_1385913240_987e326d58d1eec5428700c991f3c138.jpg resize: (3040, 4056) 1385913240 -0.14294574056802786 treat image : temp/1758878847_66244_1385913239_1e3fa5c733ac765194aecedab1d45444.jpg resize: (3040, 4056) 1385913239 -1.561070025643792 treat image : temp/1758878847_66244_1385913238_e837afd0f573bcabf618ef47fc612300.jpg resize: (3040, 4056) 1385913238 -1.5392815105878341 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 15 time used for this insertion : 0.014901161193847656 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 15 time used for this insertion : 0.014998912811279297 save missing photos in datou_result : time spend for datou_step_exec : 91.66392660140991 time spend to save output : 0.034798383712768555 total time spend for step 3 : 91.69872498512268 step4:crop_condition Fri Sep 26 11:31:59 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 15 ! batch 1 Loaded 105 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 12 About to insert : list_path_to_insert length 12 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 24 About to insert : list_path_to_insert length 24 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 17 About to insert : list_path_to_insert length 17 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1385913856, 1385913854, 1385913850, 1385913845, 1385913842, 1385913835, 1385913575, 1385913570, 1385913567, 1385913565, 1385913562, 1385913560, 1385913240, 1385913239, 1385913238] Looping around the photos to save general results len do output : 71 /-3976647761Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647771Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647778Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647784Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647796Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647799Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647801Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647805Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647809Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647810Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647821Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647841Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647786Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647788Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647812Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647814Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647825Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647835Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647843Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647848Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647775Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647780Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647803Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647854Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647762Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647764Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647768Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647769Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647774Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647777Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647783Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647789Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647794Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647802Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647806Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647811Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647819Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647823Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647826Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647834Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647839Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647844Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647846Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647849Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647855Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647856Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647860Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647862Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647759Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647760Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647763Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647765Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647772Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647779Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647781Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647782Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647792Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647797Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647800Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647820Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647824Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647827Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647828Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647840Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647851Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647795Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647816Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647822Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647830Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647836Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647787Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913856', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913854', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913850', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913845', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913842', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913835', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913575', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913570', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913567', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913565', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913562', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913560', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913240', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913239', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913238', None, None, None, None, None, '3772637') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 228 time used for this insertion : 0.04224753379821777 save_final save missing photos in datou_result : time spend for datou_step_exec : 79.218674659729 time spend to save output : 0.055451393127441406 total time spend for step 4 : 79.27412605285645 step5:thcl Fri Sep 26 11:33:18 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 1 we are using the classfication for multi_thcl [2456] time to import caffe and check if the image exist : 0.011826276779174805 time to convert the images to numpy array : 4.76837158203125e-06 time to import caffe and check if the image exist : 0.018059253692626953 time to convert the images to numpy array : 0.03611588478088379 time to import caffe and check if the image exist : 0.01923370361328125 time to convert the images to numpy array : 0.1343975067138672 time to import caffe and check if the image exist : 0.0101470947265625 time to convert the images to numpy array : 0.19997906684875488 time to import caffe and check if the image exist : 0.020558595657348633 time to convert the images to numpy array : 0.2045450210571289 time to import caffe and check if the image exist : 0.016867876052856445 time to convert the images to numpy array : 0.21033120155334473 time to import caffe and check if the image exist : 0.015424728393554688 time to convert the images to numpy array : 0.3092474937438965 time to import caffe and check if the image exist : 0.017897605895996094 time to convert the images to numpy array : 0.30948829650878906 time to import caffe and check if the image exist : 0.02237105369567871 time to convert the images to numpy array : 0.4231448173522949 time to import caffe and check if the image exist : 0.016495704650878906 time to convert the images to numpy array : 0.4389348030090332 total time to convert the images to numpy array : 0.8743133544921875 list photo_ids error: [] list photo_ids correct : [-3976647792, -3976647797, -3976647800, -3976647820, -3976647824, -3976647827, -3976647828, -3976647840, -3976647759, -3976647760, -3976647763, -3976647765, -3976647772, -3976647779, -3976647781, -3976647782, -3976647761, -3976647771, -3976647778, -3976647784, -3976647796, -3976647799, -3976647801, -3976647805, -3976647762, -3976647764, -3976647768, -3976647769, -3976647774, -3976647777, -3976647783, -3976647789, -3976647851, -3976647795, -3976647816, -3976647822, -3976647830, -3976647836, -3976647787, -3976647839, -3976647844, -3976647846, -3976647849, -3976647855, -3976647856, -3976647860, -3976647862, -3976647825, -3976647835, -3976647843, -3976647848, -3976647775, -3976647780, -3976647803, -3976647854, -3976647794, -3976647802, -3976647806, -3976647811, -3976647819, -3976647823, -3976647826, -3976647834, -3976647809, -3976647810, -3976647821, -3976647841, -3976647786, -3976647788, -3976647812, -3976647814] number of photos to traite : 71 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 5290 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/:/home/admin/workarea/git/apy/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 5071 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.3952059745788574 time used to do the prediction : 0.22748637199401855 we don't save the descriptors for this thcl 2456 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 9.775161743164062e-06 save missing photos in datou_result : time spend for datou_step_exec : 9.749154567718506 time spend to save output : 0.0002541542053222656 total time spend for step 5 : 9.749408721923828 step6:tfhub_classification2 Fri Sep 26 11:33:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou_step TFHub with tf2 ! we are using the classfication for only one thcl 3501 begin to check gpu status inside check gpu memory 2025-09-26 11:33:32.574834: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-26 11:33:32.575807: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:33:32.575905: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:33:32.575967: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:33:32.589398: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:33:32.589522: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:33:32.594431: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:33:32.595902: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:33:32.606830: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:33:32.608080: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:33:32.609037: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-26 11:33:32.636583: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-26 11:33:32.638138: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fcbcc000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-26 11:33:32.638180: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-26 11:33:32.642093: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x3e489850 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-26 11:33:32.642115: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-26 11:33:32.642990: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:33:32.643083: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:33:32.643102: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:33:32.643223: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:33:32.643243: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:33:32.643270: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:33:32.643299: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:33:32.643329: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:33:32.644219: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:33:32.644647: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:33:32.644701: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-26 11:33:32.644711: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-26 11:33:32.644719: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-26 11:33:32.645645: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3096 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) l 3637 free memory gpu now : 5069 max_wait_temp : 1 max_wait : 5 1 Physical GPUs, 1 Logical GPUs tagging for thcl : 3501 To do loadFromThcl(), then load ParamDescType : thcl3501 thcls : [{'id': 3501, 'mtr_user_id': 31, 'name': 'learn_papier_mixte_centre_tf_b4_220223_3', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '9087724,9087727,9087725,9087729,9087726,9087723,9087722,9087720,9087721,9087728', 'photo_hashtag_type': 4539, 'photo_desc_type': 5759, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 3501, 'mtr_user_id': 31, 'name': 'learn_papier_mixte_centre_tf_b4_220223_3', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '9087724,9087727,9087725,9087729,9087726,9087723,9087722,9087720,9087721,9087728', 'photo_hashtag_type': 4539, 'photo_desc_type': 5759, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5759 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5759, 'learn_papier_mixte_centre_tf_b4_220223_3', 1792, 1792, 'learn_papier_mixte_centre_tf_b4_220223_3', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 2, 22, 13, 54, 32), datetime.datetime(2023, 2, 22, 13, 54, 32)) model_name : learn_papier_mixte_centre_tf_b4_220223_3 model_param file didn't exist model_name : learn_papier_mixte_centre_tf_b4_220223_3 model_type : tf_classification2 list file need : ['Confusion_Matrix.png', 'Precision_Recall_Carton_brun.jpg', 'Precision_Recall_Carton_gris.jpg', 'Precision_Recall_Teint_Dans_La_Masse.jpg', 'Precision_Recall_autre_refus.jpg', 'Precision_Recall_cartonnette.jpg', 'Precision_Recall_environnement.jpg', 'Precision_Recall_kraft.jpg', 'Precision_Recall_metal.jpg', 'Precision_Recall_papier.jpg', 'Precision_Recall_plastique.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file exist in s3 : ['Confusion_Matrix.png', 'Precision_Recall_Carton_brun.jpg', 'Precision_Recall_Carton_gris.jpg', 'Precision_Recall_Teint_Dans_La_Masse.jpg', 'Precision_Recall_autre_refus.jpg', 'Precision_Recall_cartonnette.jpg', 'Precision_Recall_environnement.jpg', 'Precision_Recall_kraft.jpg', 'Precision_Recall_metal.jpg', 'Precision_Recall_papier.jpg', 'Precision_Recall_plastique.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file manque in s3 : [] local folder : /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3 /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Confusion_Matrix.png size_local : 121269 size in s3 : 121269 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:17 Confusion_Matrix.png already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_Carton_brun.jpg size_local : 76104 size in s3 : 76104 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_Carton_brun.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_Carton_gris.jpg size_local : 82912 size in s3 : 82912 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_Carton_gris.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_Teint_Dans_La_Masse.jpg size_local : 75423 size in s3 : 75423 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:25 Precision_Recall_Teint_Dans_La_Masse.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_autre_refus.jpg size_local : 78883 size in s3 : 78883 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_autre_refus.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_cartonnette.jpg size_local : 81791 size in s3 : 81791 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:30 Precision_Recall_cartonnette.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_environnement.jpg size_local : 59936 size in s3 : 59936 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:17 Precision_Recall_environnement.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_kraft.jpg size_local : 80280 size in s3 : 80280 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:25 Precision_Recall_kraft.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_metal.jpg size_local : 77736 size in s3 : 77736 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_metal.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_papier.jpg size_local : 69041 size in s3 : 69041 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_papier.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_plastique.jpg size_local : 70770 size in s3 : 70770 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:17 Precision_Recall_plastique.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Result_Summary.txt size_local : 1092 size in s3 : 1092 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Result_Summary.txt already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/checkpoint size_local : 99 size in s3 : 99 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 checkpoint already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_checkpoint.ckpt.data-00000-of-00002 size_local : 429801 size in s3 : 429801 create time local : 2023-03-03 12:26:16 create time in s3 : 2023-02-22 12:54:30 model_checkpoint.ckpt.data-00000-of-00002 already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_checkpoint.ckpt.data-00001-of-00002 size_local : 141033188 size in s3 : 141033188 create time local : 2023-03-03 12:26:17 create time in s3 : 2023-02-22 12:54:25 model_checkpoint.ckpt.data-00001-of-00002 already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_checkpoint.ckpt.index size_local : 85618 size in s3 : 85618 create time local : 2023-03-03 12:26:17 create time in s3 : 2023-02-22 12:54:30 model_checkpoint.ckpt.index already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_weights.h5 size_local : 71289448 size in s3 : 71289448 create time local : 2023-03-03 12:26:18 create time in s3 : 2023-02-22 12:54:17 model_weights.h5 already exist and didn't need to update desc size : 1792 Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= module (KerasLayer) (None, 1792) 17673816 _________________________________________________________________ learn_papier_mixte_centre_tf (None, 10) 17930 ================================================================= Total params: 17,691,746 Trainable params: 17,930 Non-trainable params: 17,673,816 _________________________________________________________________ Loading Weights... time used to create the model : 17.82525134086609 time used to load_weights : 0.3620929718017578 0it [00:00, ?it/s] 71it [00:00, 876.01it/s]2025-09-26 11:33:54.241022: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:33:54.245783: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 temp/1758878847_66244_1385913856_9ca0ded2129325c941dde24db09235cd_rle_crop_3976647761_0.png temp/1758878847_66244_1385913854_5999739dd5bdd4e97fd96ac1b6fbea46_rle_crop_3976647771_0.png temp/1758878847_66244_1385913850_5c29c024123e4053bb68c7de8bed0a8b_rle_crop_3976647778_0.png temp/1758878847_66244_1385913850_5c29c024123e4053bb68c7de8bed0a8b_rle_crop_3976647784_0.png temp/1758878847_66244_1385913845_8ddd20ac8b2e903721f9b368b8b0b097_rle_crop_3976647796_0.png temp/1758878847_66244_1385913842_6b30039522930f28a772668cc77bbce2_rle_crop_3976647799_0.png temp/1758878847_66244_1385913842_6b30039522930f28a772668cc77bbce2_rle_crop_3976647801_0.png temp/1758878847_66244_1385913842_6b30039522930f28a772668cc77bbce2_rle_crop_3976647805_0.png temp/1758878847_66244_1385913835_ad181f5309a1f7cd9e27d672f083f663_rle_crop_3976647809_0.png temp/1758878847_66244_1385913835_ad181f5309a1f7cd9e27d672f083f663_rle_crop_3976647810_0.png temp/1758878847_66244_1385913575_a65ca1ebf8465eec79263ecba4eff51d_rle_crop_3976647821_0.png temp/1758878847_66244_1385913567_8cb6c19bc32f74567ae805dd8556d888_rle_crop_3976647841_0.png temp/1758878847_66244_1385913850_5c29c024123e4053bb68c7de8bed0a8b_rle_crop_3976647786_0.png temp/1758878847_66244_1385913850_5c29c024123e4053bb68c7de8bed0a8b_rle_crop_3976647788_0.png temp/1758878847_66244_1385913835_ad181f5309a1f7cd9e27d672f083f663_rle_crop_3976647812_0.png temp/1758878847_66244_1385913835_ad181f5309a1f7cd9e27d672f083f663_rle_crop_3976647814_0.png temp/1758878847_66244_1385913570_8eb6b902d3d096045fe7139afe9dd77b_rle_crop_3976647825_0.png temp/1758878847_66244_1385913570_8eb6b902d3d096045fe7139afe9dd77b_rle_crop_3976647835_0.png temp/1758878847_66244_1385913567_8cb6c19bc32f74567ae805dd8556d888_rle_crop_3976647843_0.png temp/1758878847_66244_1385913565_b52eedcc066c4b070e1565313f2a95d4_rle_crop_3976647848_0.png temp/1758878847_66244_1385913854_5999739dd5bdd4e97fd96ac1b6fbea46_rle_crop_3976647775_0.png temp/1758878847_66244_1385913850_5c29c024123e4053bb68c7de8bed0a8b_rle_crop_3976647780_0.png temp/1758878847_66244_1385913842_6b30039522930f28a772668cc77bbce2_rle_crop_3976647803_0.png temp/1758878847_66244_1385913562_4d6d805dee73e40f25d8908cbf09c0a1_rle_crop_3976647854_0.png temp/1758878847_66244_1385913856_9ca0ded2129325c941dde24db09235cd_rle_crop_3976647762_0.png temp/1758878847_66244_1385913856_9ca0ded2129325c941dde24db09235cd_rle_crop_3976647764_0.png temp/1758878847_66244_1385913856_9ca0ded2129325c941dde24db09235cd_rle_crop_3976647768_0.png temp/1758878847_66244_1385913856_9ca0ded2129325c941dde24db09235cd_rle_crop_3976647769_0.png temp/1758878847_66244_1385913854_5999739dd5bdd4e97fd96ac1b6fbea46_rle_crop_3976647774_0.png temp/1758878847_66244_1385913854_5999739dd5bdd4e97fd96ac1b6fbea46_rle_crop_3976647777_0.png temp/1758878847_66244_1385913850_5c29c024123e4053bb68c7de8bed0a8b_rle_crop_3976647783_0.png temp/1758878847_66244_1385913850_5c29c024123e4053bb68c7de8bed0a8b_rle_crop_3976647789_0.png temp/1758878847_66244_1385913845_8ddd20ac8b2e903721f9b368b8b0b097_rle_crop_3976647794_0.png temp/1758878847_66244_1385913842_6b30039522930f28a772668cc77bbce2_rle_crop_3976647802_0.png temp/1758878847_66244_1385913842_6b30039522930f28a772668cc77bbce2_rle_crop_3976647806_0.png temp/1758878847_66244_1385913835_ad181f5309a1f7cd9e27d672f083f663_rle_crop_3976647811_0.png temp/1758878847_66244_1385913575_a65ca1ebf8465eec79263ecba4eff51d_rle_crop_3976647819_0.png temp/1758878847_66244_1385913575_a65ca1ebf8465eec79263ecba4eff51d_rle_crop_3976647823_0.png temp/1758878847_66244_1385913570_8eb6b902d3d096045fe7139afe9dd77b_rle_crop_3976647826_0.png temp/1758878847_66244_1385913570_8eb6b902d3d096045fe7139afe9dd77b_rle_crop_3976647834_0.png temp/1758878847_66244_1385913567_8cb6c19bc32f74567ae805dd8556d888_rle_crop_3976647839_0.png temp/1758878847_66244_1385913567_8cb6c19bc32f74567ae805dd8556d888_rle_crop_3976647844_0.png temp/1758878847_66244_1385913565_b52eedcc066c4b070e1565313f2a95d4_rle_crop_3976647846_0.png temp/1758878847_66244_1385913565_b52eedcc066c4b070e1565313f2a95d4_rle_crop_3976647849_0.png temp/1758878847_66244_1385913562_4d6d805dee73e40f25d8908cbf09c0a1_rle_crop_3976647855_0.png temp/1758878847_66244_1385913562_4d6d805dee73e40f25d8908cbf09c0a1_rle_crop_3976647856_0.png temp/1758878847_66244_1385913240_987e326d58d1eec5428700c991f3c138_rle_crop_3976647860_0.png temp/1758878847_66244_1385913239_1e3fa5c733ac765194aecedab1d45444_rle_crop_3976647862_0.png temp/1758878847_66244_1385913856_9ca0ded2129325c941dde24db09235cd_rle_crop_3976647759_0.png temp/1758878847_66244_1385913856_9ca0ded2129325c941dde24db09235cd_rle_crop_3976647760_0.png temp/1758878847_66244_1385913856_9ca0ded2129325c941dde24db09235cd_rle_crop_3976647763_0.png temp/1758878847_66244_1385913856_9ca0ded2129325c941dde24db09235cd_rle_crop_3976647765_0.png temp/1758878847_66244_1385913854_5999739dd5bdd4e97fd96ac1b6fbea46_rle_crop_3976647772_0.png temp/1758878847_66244_1385913850_5c29c024123e4053bb68c7de8bed0a8b_rle_crop_3976647779_0.png temp/1758878847_66244_1385913850_5c29c024123e4053bb68c7de8bed0a8b_rle_crop_3976647781_0.png temp/1758878847_66244_1385913850_5c29c024123e4053bb68c7de8bed0a8b_rle_crop_3976647782_0.png temp/1758878847_66244_1385913845_8ddd20ac8b2e903721f9b368b8b0b097_rle_crop_3976647792_0.png temp/1758878847_66244_1385913845_8ddd20ac8b2e903721f9b368b8b0b097_rle_crop_3976647797_0.png temp/1758878847_66244_1385913842_6b30039522930f28a772668cc77bbce2_rle_crop_3976647800_0.png temp/1758878847_66244_1385913575_a65ca1ebf8465eec79263ecba4eff51d_rle_crop_3976647820_0.png temp/1758878847_66244_1385913575_a65ca1ebf8465eec79263ecba4eff51d_rle_crop_3976647824_0.png temp/1758878847_66244_1385913570_8eb6b902d3d096045fe7139afe9dd77b_rle_crop_3976647827_0.png temp/1758878847_66244_1385913570_8eb6b902d3d096045fe7139afe9dd77b_rle_crop_3976647828_0.png temp/1758878847_66244_1385913567_8cb6c19bc32f74567ae805dd8556d888_rle_crop_3976647840_0.png temp/1758878847_66244_1385913562_4d6d805dee73e40f25d8908cbf09c0a1_rle_crop_3976647851_0.png temp/1758878847_66244_1385913845_8ddd20ac8b2e903721f9b368b8b0b097_rle_crop_3976647795_0.png temp/1758878847_66244_1385913835_ad181f5309a1f7cd9e27d672f083f663_rle_crop_3976647816_0.png temp/1758878847_66244_1385913575_a65ca1ebf8465eec79263ecba4eff51d_rle_crop_3976647822_0.png temp/1758878847_66244_1385913570_8eb6b902d3d096045fe7139afe9dd77b_rle_crop_3976647830_0.png temp/1758878847_66244_1385913570_8eb6b902d3d096045fe7139afe9dd77b_rle_crop_3976647836_0.png temp/1758878847_66244_1385913850_5c29c024123e4053bb68c7de8bed0a8b_rle_crop_3976647787_0.png Found 71 images belonging to 1 classes. begin to do the prediction : time used to do the prediction : 5.460433006286621 (71,) (71, 10) (71, 1792) shape of features : (71, 1792) shape of new features : (1, 71, 1792) save descriptor for thcl : 3501 time to traite the descriptors : 0.3918302059173584 storage_type for insertDescriptorsMulti : 3 Missing photo l117 : -3976647761 Missing photo l117 : -3976647771 Missing photo l117 : -3976647778 Missing photo l117 : -3976647784 Missing photo l117 : -3976647796 Missing photo l117 : -3976647799 Missing photo l117 : -3976647801 Missing photo l117 : -3976647805 Missing photo l117 : -3976647809 Missing photo l117 : -3976647810 Missing photo l117 : -3976647821 Missing photo l117 : -3976647841 Missing photo l117 : -3976647786 Missing photo l117 : -3976647788 Missing photo l117 : -3976647812 Missing photo l117 : -3976647814 Missing photo l117 : -3976647825 Missing photo l117 : -3976647835 Missing photo l117 : -3976647843 Missing photo l117 : -3976647848 Missing photo l117 : -3976647775 Missing photo l117 : -3976647780 Missing photo l117 : -3976647803 Missing photo l117 : -3976647854 Missing photo l117 : -3976647762 Missing photo l117 : -3976647764 Missing photo l117 : -3976647768 Missing photo l117 : -3976647769 Missing photo l117 : -3976647774 Missing photo l117 : -3976647777 Missing photo l117 : -3976647783 Missing photo l117 : -3976647789 Missing photo l117 : -3976647794 Missing photo l117 : -3976647802 Missing photo l117 : -3976647806 Missing photo l117 : -3976647811 Missing photo l117 : -3976647819 Missing photo l117 : -3976647823 Missing photo l117 : -3976647826 Missing photo l117 : -3976647834 Missing photo l117 : -3976647839 Missing photo l117 : -3976647844 Missing photo l117 : -3976647846 Missing photo l117 : -3976647849 Missing photo l117 : -3976647855 Missing photo l117 : -3976647856 Missing photo l117 : -3976647860 Missing photo l117 : -3976647862 Missing photo l117 : -3976647759 Missing photo l117 : -3976647760 Missing photo l117 : -3976647763 Missing photo l117 : -3976647765 Missing photo l117 : -3976647772 Missing photo l117 : -3976647779 Missing photo l117 : -3976647781 Missing photo l117 : -3976647782 Missing photo l117 : -3976647792 Missing photo l117 : -3976647797 Missing photo l117 : -3976647800 Missing photo l117 : -3976647820 Missing photo l117 : -3976647824 Missing photo l117 : -3976647827 Missing photo l117 : -3976647828 Missing photo l117 : -3976647840 Missing photo l117 : -3976647851 Missing photo l117 : -3976647795 Missing photo l117 : -3976647816 Missing photo l117 : -3976647822 Missing photo l117 : -3976647830 Missing photo l117 : -3976647836 Missing photo l117 : -3976647787 To insert : -3976647761 Missing photo l134 : -3976647761 To insert : -3976647771 Missing photo l134 : -3976647771 To insert : -3976647778 Missing photo l134 : -3976647778 To insert : -3976647784 Missing photo l134 : -3976647784 To insert : -3976647796 Missing photo l134 : -3976647796 To insert : -3976647799 Missing photo l134 : -3976647799 To insert : -3976647801 Missing photo l134 : -3976647801 To insert : -3976647805 Missing photo l134 : -3976647805 To insert : -3976647809 Missing photo l134 : -3976647809 To insert : -3976647810 Missing photo l134 : -3976647810 To insert : -3976647821 Missing photo l134 : -3976647821 To insert : -3976647841 Missing photo l134 : -3976647841 To insert : -3976647786 Missing photo l134 : -3976647786 To insert : -3976647788 Missing photo l134 : -3976647788 To insert : -3976647812 Missing photo l134 : -3976647812 To insert : -3976647814 Missing photo l134 : -3976647814 To insert : -3976647825 Missing photo l134 : -3976647825 To insert : -3976647835 Missing photo l134 : -3976647835 To insert : -3976647843 Missing photo l134 : -3976647843 To insert : -3976647848 Missing photo l134 : -3976647848 To insert : -3976647775 Missing photo l134 : -3976647775 To insert : -3976647780 Missing photo l134 : -3976647780 To insert : -3976647803 Missing photo l134 : -3976647803 To insert : -3976647854 Missing photo l134 : -3976647854 To insert : -3976647762 Missing photo l134 : -3976647762 To insert : -3976647764 Missing photo l134 : -3976647764 To insert : -3976647768 Missing photo l134 : -3976647768 To insert : -3976647769 Missing photo l134 : -3976647769 To insert : -3976647774 Missing photo l134 : -3976647774 To insert : -3976647777 Missing photo l134 : -3976647777 To insert : -3976647783 Missing photo l134 : -3976647783 To insert : -3976647789 Missing photo l134 : -3976647789 To insert : -3976647794 Missing photo l134 : -3976647794 To insert : -3976647802 Missing photo l134 : -3976647802 To insert : -3976647806 Missing photo l134 : -3976647806 To insert : -3976647811 Missing photo l134 : -3976647811 To insert : -3976647819 Missing photo l134 : -3976647819 To insert : -3976647823 Missing photo l134 : -3976647823 To insert : -3976647826 Missing photo l134 : -3976647826 To insert : -3976647834 Missing photo l134 : -3976647834 To insert : -3976647839 Missing photo l134 : -3976647839 To insert : -3976647844 Missing photo l134 : -3976647844 To insert : -3976647846 Missing photo l134 : -3976647846 To insert : -3976647849 Missing photo l134 : -3976647849 To insert : -3976647855 Missing photo l134 : -3976647855 To insert : -3976647856 Missing photo l134 : -3976647856 To insert : -3976647860 Missing photo l134 : -3976647860 To insert : -3976647862 Missing photo l134 : -3976647862 To insert : -3976647759 Missing photo l134 : -3976647759 To insert : -3976647760 Missing photo l134 : -3976647760 To insert : -3976647763 Missing photo l134 : -3976647763 To insert : -3976647765 Missing photo l134 : -3976647765 To insert : -3976647772 Missing photo l134 : -3976647772 To insert : -3976647779 Missing photo l134 : -3976647779 To insert : -3976647781 Missing photo l134 : -3976647781 To insert : -3976647782 Missing photo l134 : -3976647782 To insert : -3976647792 Missing photo l134 : -3976647792 To insert : -3976647797 Missing photo l134 : -3976647797 To insert : -3976647800 Missing photo l134 : -3976647800 To insert : -3976647820 Missing photo l134 : -3976647820 To insert : -3976647824 Missing photo l134 : -3976647824 To insert : -3976647827 Missing photo l134 : -3976647827 To insert : -3976647828 Missing photo l134 : -3976647828 To insert : -3976647840 Missing photo l134 : -3976647840 To insert : -3976647851 Missing photo l134 : -3976647851 To insert : -3976647795 Missing photo l134 : -3976647795 To insert : -3976647816 Missing photo l134 : -3976647816 To insert : -3976647822 Missing photo l134 : -3976647822 To insert : -3976647830 Missing photo l134 : -3976647830 To insert : -3976647836 Missing photo l134 : -3976647836 To insert : -3976647787 Missing photo l134 : -3976647787 time to insert the descriptors : 20.819822788238525 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : tfhub_classification2 we use saveGeneral [1385913856, 1385913854, 1385913850, 1385913845, 1385913842, 1385913835, 1385913575, 1385913570, 1385913567, 1385913565, 1385913562, 1385913560, 1385913240, 1385913239, 1385913238] Looping around the photos to save general results len do output : 71 /-3976647761Didn't retrieve data . /-3976647771Didn't retrieve data . /-3976647778Didn't retrieve data . /-3976647784Didn't retrieve data . /-3976647796Didn't retrieve data . /-3976647799Didn't retrieve data . /-3976647801Didn't retrieve data . /-3976647805Didn't retrieve data . /-3976647809Didn't retrieve data . /-3976647810Didn't retrieve data . /-3976647821Didn't retrieve data . /-3976647841Didn't retrieve data . /-3976647786Didn't retrieve data . /-3976647788Didn't retrieve data . /-3976647812Didn't retrieve data . /-3976647814Didn't retrieve data . /-3976647825Didn't retrieve data . /-3976647835Didn't retrieve data . /-3976647843Didn't retrieve data . /-3976647848Didn't retrieve data . /-3976647775Didn't retrieve data . /-3976647780Didn't retrieve data . /-3976647803Didn't retrieve data . /-3976647854Didn't retrieve data . /-3976647762Didn't retrieve data . /-3976647764Didn't retrieve data . /-3976647768Didn't retrieve data . /-3976647769Didn't retrieve data . /-3976647774Didn't retrieve data . /-3976647777Didn't retrieve data . /-3976647783Didn't retrieve data . /-3976647789Didn't retrieve data . /-3976647794Didn't retrieve data . /-3976647802Didn't retrieve data . /-3976647806Didn't retrieve data . /-3976647811Didn't retrieve data . /-3976647819Didn't retrieve data . /-3976647823Didn't retrieve data . /-3976647826Didn't retrieve data . /-3976647834Didn't retrieve data . /-3976647839Didn't retrieve data . /-3976647844Didn't retrieve data . /-3976647846Didn't retrieve data . /-3976647849Didn't retrieve data . /-3976647855Didn't retrieve data . /-3976647856Didn't retrieve data . /-3976647860Didn't retrieve data . /-3976647862Didn't retrieve data . /-3976647759Didn't retrieve data . /-3976647760Didn't retrieve data . /-3976647763Didn't retrieve data . /-3976647765Didn't retrieve data . /-3976647772Didn't retrieve data . /-3976647779Didn't retrieve data . /-3976647781Didn't retrieve data . /-3976647782Didn't retrieve data . /-3976647792Didn't retrieve data . /-3976647797Didn't retrieve data . /-3976647800Didn't retrieve data . /-3976647820Didn't retrieve data . /-3976647824Didn't retrieve data . /-3976647827Didn't retrieve data . /-3976647828Didn't retrieve data . /-3976647840Didn't retrieve data . /-3976647851Didn't retrieve data . /-3976647795Didn't retrieve data . /-3976647816Didn't retrieve data . /-3976647822Didn't retrieve data . /-3976647830Didn't retrieve data . /-3976647836Didn't retrieve data . /-3976647787Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913856', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913854', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913850', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913845', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913842', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913835', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913575', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913570', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913567', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913565', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913562', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913560', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913240', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913239', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913238', None, None, None, None, None, '3772637') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 86 time used for this insertion : 0.026726961135864258 save_final save missing photos in datou_result : time spend for datou_step_exec : 52.611239194869995 time spend to save output : 0.297027587890625 total time spend for step 6 : 52.90826678276062 step7:merge_mask_thcl_custom Fri Sep 26 11:34:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 105 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1385913856, 1385913854, 1385913850, 1385913845, 1385913842, 1385913835, 1385913575, 1385913570, 1385913567, 1385913565, 1385913562, 1385913560, 1385913240, 1385913239, 1385913238] Looping around the photos to save general results len do output : 15 /1385913856Didn't retrieve data .Didn't retrieve data . /1385913854Didn't retrieve data .Didn't retrieve data . /1385913850Didn't retrieve data .Didn't retrieve data . /1385913845Didn't retrieve data .Didn't retrieve data . /1385913842Didn't retrieve data .Didn't retrieve data . /1385913835Didn't retrieve data .Didn't retrieve data . /1385913575Didn't retrieve data .Didn't retrieve data . /1385913570Didn't retrieve data .Didn't retrieve data . /1385913567Didn't retrieve data .Didn't retrieve data . /1385913565Didn't retrieve data .Didn't retrieve data . /1385913562Didn't retrieve data .Didn't retrieve data . /1385913560Didn't retrieve data .Didn't retrieve data . /1385913240Didn't retrieve data .Didn't retrieve data . /1385913239Didn't retrieve data .Didn't retrieve data . /1385913238Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913856', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913854', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913850', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913845', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913842', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913835', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913575', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913570', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913567', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913565', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913562', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913560', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913240', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913239', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913238', None, None, None, None, None, '3772637') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 45 time used for this insertion : 0.014084100723266602 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.06472301483154297 time spend to save output : 0.014812707901000977 total time spend for step 7 : 0.07953572273254395 step8:rle_unique_nms_with_priority Fri Sep 26 11:34:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 11 nb_hashtags : 5 time to prepare the origin masks : 11.879345178604126 time for calcul the mask position with numpy : 0.6247069835662842 nb_pixel_total : 10252606 time to create 1 rle with new method : 2.430532217025757 time for calcul the mask position with numpy : 0.06007671356201172 nb_pixel_total : 432042 time to create 1 rle with new method : 1.2644131183624268 time for calcul the mask position with numpy : 0.052190303802490234 nb_pixel_total : 245 time to create 1 rle with old method : 0.0005333423614501953 time for calcul the mask position with numpy : 0.050493717193603516 nb_pixel_total : 49059 time to create 1 rle with old method : 0.060291290283203125 time for calcul the mask position with numpy : 0.0637352466583252 nb_pixel_total : 97753 time to create 1 rle with old method : 0.10481047630310059 time for calcul the mask position with numpy : 0.06012678146362305 nb_pixel_total : 32002 time to create 1 rle with old method : 0.05087471008300781 time for calcul the mask position with numpy : 0.03900504112243652 nb_pixel_total : 65419 time to create 1 rle with old method : 0.07079100608825684 time for calcul the mask position with numpy : 0.04299044609069824 nb_pixel_total : 119626 time to create 1 rle with old method : 0.13788747787475586 time for calcul the mask position with numpy : 0.04486203193664551 nb_pixel_total : 6898 time to create 1 rle with old method : 0.007746219635009766 time for calcul the mask position with numpy : 0.057816267013549805 nb_pixel_total : 1033789 time to create 1 rle with new method : 1.2200124263763428 time for calcul the mask position with numpy : 0.03966259956359863 nb_pixel_total : 92989 time to create 1 rle with old method : 0.10420799255371094 time for calcul the mask position with numpy : 0.05523347854614258 nb_pixel_total : 147812 time to create 1 rle with old method : 0.15981364250183105 create new chi : 6.948254346847534 time to delete rle : 0.02695178985595703 batch 1 Loaded 12 chid ids of type : 3726 Number RLEs to save : 16012 TO DO : save crop sub photo not yet done ! save time : 1.1611170768737793 nb_obj : 6 nb_hashtags : 4 time to prepare the origin masks : 9.875333070755005 time for calcul the mask position with numpy : 0.7380189895629883 nb_pixel_total : 10818458 time to create 1 rle with new method : 1.0255870819091797 time for calcul the mask position with numpy : 0.03759193420410156 nb_pixel_total : 118416 time to create 1 rle with old method : 0.12676048278808594 time for calcul the mask position with numpy : 0.04660749435424805 nb_pixel_total : 1206437 time to create 1 rle with new method : 1.5428028106689453 time for calcul the mask position with numpy : 0.05074286460876465 nb_pixel_total : 37508 time to create 1 rle with old method : 0.05763959884643555 time for calcul the mask position with numpy : 0.040593624114990234 nb_pixel_total : 7720 time to create 1 rle with old method : 0.009215593338012695 time for calcul the mask position with numpy : 0.04141736030578613 nb_pixel_total : 135247 time to create 1 rle with old method : 0.16058063507080078 time for calcul the mask position with numpy : 0.046228647232055664 nb_pixel_total : 6454 time to create 1 rle with old method : 0.012211799621582031 create new chi : 4.0523717403411865 time to delete rle : 0.0020439624786376953 batch 1 Loaded 7 chid ids of type : 3726 Number RLEs to save : 13130 TO DO : save crop sub photo not yet done ! save time : 0.7818303108215332 nb_obj : 13 nb_hashtags : 7 time to prepare the origin masks : 15.387531280517578 time for calcul the mask position with numpy : 0.7741928100585938 nb_pixel_total : 9967795 time to create 1 rle with new method : 0.7935411930084229 time for calcul the mask position with numpy : 0.0820622444152832 nb_pixel_total : 33992 time to create 1 rle with old method : 0.040657758712768555 time for calcul the mask position with numpy : 0.06839609146118164 nb_pixel_total : 159780 time to create 1 rle with new method : 0.6493339538574219 time for calcul the mask position with numpy : 0.04480171203613281 nb_pixel_total : 36384 time to create 1 rle with old method : 0.04611611366271973 time for calcul the mask position with numpy : 0.05256295204162598 nb_pixel_total : 121599 time to create 1 rle with old method : 0.14800286293029785 time for calcul the mask position with numpy : 0.040741682052612305 nb_pixel_total : 362317 time to create 1 rle with new method : 0.7046496868133545 time for calcul the mask position with numpy : 0.04007101058959961 nb_pixel_total : 111299 time to create 1 rle with old method : 0.12378501892089844 time for calcul the mask position with numpy : 0.043958425521850586 nb_pixel_total : 16427 time to create 1 rle with old method : 0.01834583282470703 time for calcul the mask position with numpy : 0.04027104377746582 nb_pixel_total : 11191 time to create 1 rle with old method : 0.012597084045410156 time for calcul the mask position with numpy : 0.041410207748413086 nb_pixel_total : 132819 time to create 1 rle with old method : 0.14689350128173828 time for calcul the mask position with numpy : 0.04921841621398926 nb_pixel_total : 1199634 time to create 1 rle with new method : 0.726301908493042 time for calcul the mask position with numpy : 0.03839755058288574 nb_pixel_total : 4213 time to create 1 rle with old method : 0.005357503890991211 time for calcul the mask position with numpy : 0.04169058799743652 nb_pixel_total : 50611 time to create 1 rle with old method : 0.05668139457702637 time for calcul the mask position with numpy : 0.03911328315734863 nb_pixel_total : 122179 time to create 1 rle with old method : 0.13662433624267578 create new chi : 5.210409164428711 time to delete rle : 0.002337932586669922 batch 1 Loaded 14 chid ids of type : 3726 Number RLEs to save : 16510 TO DO : save crop sub photo not yet done ! save time : 1.2453148365020752 nb_obj : 6 nb_hashtags : 4 time to prepare the origin masks : 9.15471076965332 time for calcul the mask position with numpy : 1.1929125785827637 nb_pixel_total : 11466642 time to create 1 rle with new method : 1.2775518894195557 time for calcul the mask position with numpy : 0.03813481330871582 nb_pixel_total : 74159 time to create 1 rle with old method : 0.0883631706237793 time for calcul the mask position with numpy : 0.03737187385559082 nb_pixel_total : 39096 time to create 1 rle with old method : 0.04903125762939453 time for calcul the mask position with numpy : 0.03901052474975586 nb_pixel_total : 9427 time to create 1 rle with old method : 0.01071786880493164 time for calcul the mask position with numpy : 0.03798079490661621 nb_pixel_total : 30642 time to create 1 rle with old method : 0.03485727310180664 time for calcul the mask position with numpy : 0.040219783782958984 nb_pixel_total : 18505 time to create 1 rle with old method : 0.021001577377319336 time for calcul the mask position with numpy : 0.1061549186706543 nb_pixel_total : 691769 time to create 1 rle with new method : 1.3533225059509277 create new chi : 4.4558165073394775 time to delete rle : 0.0015625953674316406 batch 1 Loaded 7 chid ids of type : 3726 Number RLEs to save : 9875 TO DO : save crop sub photo not yet done ! save time : 0.6055197715759277 nb_obj : 9 nb_hashtags : 5 time to prepare the origin masks : 13.785295963287354 time for calcul the mask position with numpy : 0.675046443939209 nb_pixel_total : 10490190 time to create 1 rle with new method : 1.067559003829956 time for calcul the mask position with numpy : 0.0764925479888916 nb_pixel_total : 198668 time to create 1 rle with new method : 0.9621987342834473 time for calcul the mask position with numpy : 0.051302433013916016 nb_pixel_total : 224269 time to create 1 rle with new method : 0.6100735664367676 time for calcul the mask position with numpy : 0.03931093215942383 nb_pixel_total : 27859 time to create 1 rle with old method : 0.03643178939819336 time for calcul the mask position with numpy : 0.04169273376464844 nb_pixel_total : 149848 time to create 1 rle with old method : 0.18018627166748047 time for calcul the mask position with numpy : 0.05811929702758789 nb_pixel_total : 35685 time to create 1 rle with old method : 0.04048562049865723 time for calcul the mask position with numpy : 0.06161999702453613 nb_pixel_total : 48483 time to create 1 rle with old method : 0.05292677879333496 time for calcul the mask position with numpy : 0.10772204399108887 nb_pixel_total : 239512 time to create 1 rle with new method : 0.7365789413452148 time for calcul the mask position with numpy : 0.07221150398254395 nb_pixel_total : 897405 time to create 1 rle with new method : 0.6076838970184326 time for calcul the mask position with numpy : 0.0593571662902832 nb_pixel_total : 18321 time to create 1 rle with old method : 0.01929163932800293 create new chi : 5.780999422073364 time to delete rle : 0.0016415119171142578 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 15044 TO DO : save crop sub photo not yet done ! save time : 0.9311802387237549 nb_obj : 8 nb_hashtags : 5 time to prepare the origin masks : 12.220351696014404 time for calcul the mask position with numpy : 0.553187370300293 nb_pixel_total : 10240845 time to create 1 rle with new method : 0.9404306411743164 time for calcul the mask position with numpy : 0.04892730712890625 nb_pixel_total : 381033 time to create 1 rle with new method : 0.9653275012969971 time for calcul the mask position with numpy : 0.04760289192199707 nb_pixel_total : 89676 time to create 1 rle with old method : 0.1059257984161377 time for calcul the mask position with numpy : 0.07052278518676758 nb_pixel_total : 30434 time to create 1 rle with old method : 0.05479121208190918 time for calcul the mask position with numpy : 0.07629728317260742 nb_pixel_total : 85087 time to create 1 rle with old method : 0.09671831130981445 time for calcul the mask position with numpy : 0.07284975051879883 nb_pixel_total : 271198 time to create 1 rle with new method : 0.7765040397644043 time for calcul the mask position with numpy : 0.07308197021484375 nb_pixel_total : 377389 time to create 1 rle with new method : 1.4382846355438232 time for calcul the mask position with numpy : 0.07639336585998535 nb_pixel_total : 4271 time to create 1 rle with old method : 0.0051081180572509766 time for calcul the mask position with numpy : 0.09252715110778809 nb_pixel_total : 850307 time to create 1 rle with new method : 1.1801707744598389 create new chi : 6.9453840255737305 time to delete rle : 0.0028793811798095703 batch 1 Loaded 9 chid ids of type : 3726 Number RLEs to save : 16369 TO DO : save crop sub photo not yet done ! save time : 0.9760429859161377 nb_obj : 6 nb_hashtags : 3 time to prepare the origin masks : 10.689902067184448 time for calcul the mask position with numpy : 0.5716075897216797 nb_pixel_total : 11404501 time to create 1 rle with new method : 1.522986888885498 time for calcul the mask position with numpy : 0.07289695739746094 nb_pixel_total : 296645 time to create 1 rle with new method : 0.9687705039978027 time for calcul the mask position with numpy : 0.06478095054626465 nb_pixel_total : 67732 time to create 1 rle with old method : 0.07799291610717773 time for calcul the mask position with numpy : 0.07257986068725586 nb_pixel_total : 234880 time to create 1 rle with new method : 1.362471103668213 time for calcul the mask position with numpy : 0.07150936126708984 nb_pixel_total : 296471 time to create 1 rle with new method : 0.9216382503509521 time for calcul the mask position with numpy : 0.061205148696899414 nb_pixel_total : 21075 time to create 1 rle with old method : 0.02330303192138672 time for calcul the mask position with numpy : 0.055819034576416016 nb_pixel_total : 8936 time to create 1 rle with old method : 0.013153314590454102 create new chi : 6.07342529296875 time to delete rle : 0.0017910003662109375 batch 1 Loaded 7 chid ids of type : 3726 Number RLEs to save : 8562 TO DO : save crop sub photo not yet done ! save time : 0.5474722385406494 nb_obj : 9 nb_hashtags : 5 time to prepare the origin masks : 11.269398927688599 time for calcul the mask position with numpy : 1.0827643871307373 nb_pixel_total : 9997212 time to create 1 rle with new method : 1.409069538116455 time for calcul the mask position with numpy : 0.1036074161529541 nb_pixel_total : 71365 time to create 1 rle with old method : 0.0802149772644043 time for calcul the mask position with numpy : 0.07737064361572266 nb_pixel_total : 460159 time to create 1 rle with new method : 1.1323339939117432 time for calcul the mask position with numpy : 0.09604406356811523 nb_pixel_total : 17120 time to create 1 rle with old method : 0.019716262817382812 time for calcul the mask position with numpy : 0.07313370704650879 nb_pixel_total : 59908 time to create 1 rle with old method : 0.06347393989562988 time for calcul the mask position with numpy : 0.07499575614929199 nb_pixel_total : 7234 time to create 1 rle with old method : 0.008014917373657227 time for calcul the mask position with numpy : 0.08046174049377441 nb_pixel_total : 977627 time to create 1 rle with new method : 1.2176992893218994 time for calcul the mask position with numpy : 0.06680798530578613 nb_pixel_total : 190221 time to create 1 rle with new method : 0.9902069568634033 time for calcul the mask position with numpy : 0.07492327690124512 nb_pixel_total : 523824 time to create 1 rle with new method : 1.0941050052642822 time for calcul the mask position with numpy : 0.09056258201599121 nb_pixel_total : 25570 time to create 1 rle with old method : 0.026937246322631836 create new chi : 8.165029287338257 time to delete rle : 0.0032176971435546875 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 15152 TO DO : save crop sub photo not yet done ! save time : 0.9336252212524414 nb_obj : 6 nb_hashtags : 3 time to prepare the origin masks : 10.281544208526611 time for calcul the mask position with numpy : 1.034268856048584 nb_pixel_total : 10488872 time to create 1 rle with new method : 1.211869239807129 time for calcul the mask position with numpy : 0.061022281646728516 nb_pixel_total : 112959 time to create 1 rle with old method : 0.11586737632751465 time for calcul the mask position with numpy : 0.06137585639953613 nb_pixel_total : 27533 time to create 1 rle with old method : 0.028834819793701172 time for calcul the mask position with numpy : 0.04519319534301758 nb_pixel_total : 66491 time to create 1 rle with old method : 0.09102678298950195 time for calcul the mask position with numpy : 0.049379825592041016 nb_pixel_total : 16195 time to create 1 rle with old method : 0.01842784881591797 time for calcul the mask position with numpy : 0.07907271385192871 nb_pixel_total : 417233 time to create 1 rle with new method : 1.0535669326782227 time for calcul the mask position with numpy : 0.09246420860290527 nb_pixel_total : 1200957 time to create 1 rle with new method : 1.3210914134979248 create new chi : 5.4601829051971436 time to delete rle : 0.0027360916137695312 batch 1 Loaded 7 chid ids of type : 3726 Number RLEs to save : 14522 TO DO : save crop sub photo not yet done ! save time : 0.8599097728729248 nb_obj : 6 nb_hashtags : 2 time to prepare the origin masks : 10.011985540390015 time for calcul the mask position with numpy : 0.6561236381530762 nb_pixel_total : 10390440 time to create 1 rle with new method : 1.0975220203399658 time for calcul the mask position with numpy : 0.04881477355957031 nb_pixel_total : 174562 time to create 1 rle with new method : 1.448960781097412 time for calcul the mask position with numpy : 0.06277680397033691 nb_pixel_total : 157906 time to create 1 rle with new method : 1.0044820308685303 time for calcul the mask position with numpy : 0.07026982307434082 nb_pixel_total : 1244251 time to create 1 rle with new method : 1.2514963150024414 time for calcul the mask position with numpy : 0.08016061782836914 nb_pixel_total : 63028 time to create 1 rle with old method : 0.06575822830200195 time for calcul the mask position with numpy : 0.08299756050109863 nb_pixel_total : 968 time to create 1 rle with old method : 0.0010709762573242188 time for calcul the mask position with numpy : 0.08875131607055664 nb_pixel_total : 299085 time to create 1 rle with new method : 1.0405149459838867 create new chi : 7.49879264831543 time to delete rle : 0.0026302337646484375 batch 1 Loaded 7 chid ids of type : 3726 Number RLEs to save : 13920 TO DO : save crop sub photo not yet done ! save time : 0.9090657234191895 nb_obj : 7 nb_hashtags : 5 time to prepare the origin masks : 12.504854440689087 time for calcul the mask position with numpy : 1.0544853210449219 nb_pixel_total : 9179790 time to create 1 rle with new method : 1.6338729858398438 time for calcul the mask position with numpy : 0.04255080223083496 nb_pixel_total : 73833 time to create 1 rle with old method : 0.07648277282714844 time for calcul the mask position with numpy : 0.044077396392822266 nb_pixel_total : 259656 time to create 1 rle with new method : 2.0661461353302 time for calcul the mask position with numpy : 0.04893326759338379 nb_pixel_total : 512559 time to create 1 rle with new method : 1.1864814758300781 time for calcul the mask position with numpy : 0.08737611770629883 nb_pixel_total : 1305423 time to create 1 rle with new method : 0.9826979637145996 time for calcul the mask position with numpy : 0.07221293449401855 nb_pixel_total : 14277 time to create 1 rle with old method : 0.018088579177856445 time for calcul the mask position with numpy : 0.07897138595581055 nb_pixel_total : 963221 time to create 1 rle with new method : 1.4998810291290283 time for calcul the mask position with numpy : 0.1003725528717041 nb_pixel_total : 21481 time to create 1 rle with old method : 0.025708436965942383 create new chi : 9.298425436019897 time to delete rle : 0.002490520477294922 batch 1 Loaded 8 chid ids of type : 3726 Number RLEs to save : 18230 TO DO : save crop sub photo not yet done ! save time : 1.7013885974884033 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 2.2134053707122803 time for calcul the mask position with numpy : 0.6182289123535156 nb_pixel_total : 5970046 time to create 1 rle with new method : 1.6721954345703125 time for calcul the mask position with numpy : 0.5366709232330322 nb_pixel_total : 6360194 time to create 1 rle with new method : 1.1180179119110107 create new chi : 4.077076196670532 time to delete rle : 0.0018661022186279297 batch 1 Loaded 2 chid ids of type : 3726 Number RLEs to save : 12612 TO DO : save crop sub photo not yet done ! save time : 0.7723817825317383 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 7.078033447265625 time for calcul the mask position with numpy : 0.8153977394104004 nb_pixel_total : 6848676 time to create 1 rle with new method : 1.338822603225708 time for calcul the mask position with numpy : 0.0630333423614502 nb_pixel_total : 50395 time to create 1 rle with old method : 0.05800509452819824 time for calcul the mask position with numpy : 0.42442870140075684 nb_pixel_total : 5431169 time to create 1 rle with new method : 1.1078062057495117 create new chi : 3.9188578128814697 time to delete rle : 0.0019211769104003906 batch 1 Loaded 3 chid ids of type : 3726 Number RLEs to save : 15634 TO DO : save crop sub photo not yet done ! save time : 0.9804782867431641 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 5.5663697719573975 time for calcul the mask position with numpy : 0.08518838882446289 nb_pixel_total : 2694552 time to create 1 rle with new method : 2.059353828430176 time for calcul the mask position with numpy : 0.06527900695800781 nb_pixel_total : 69317 time to create 1 rle with old method : 0.08337879180908203 time for calcul the mask position with numpy : 0.8858442306518555 nb_pixel_total : 9566371 time to create 1 rle with new method : 1.0012352466583252 create new chi : 4.296262741088867 time to delete rle : 0.002306699752807617 batch 1 Loaded 3 chid ids of type : 3726 Number RLEs to save : 9632 TO DO : save crop sub photo not yet done ! save time : 0.6161150932312012 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 2.4635186195373535 time for calcul the mask position with numpy : 0.07843899726867676 nb_pixel_total : 4488644 time to create 1 rle with new method : 0.8259749412536621 time for calcul the mask position with numpy : 0.48576855659484863 nb_pixel_total : 7841596 time to create 1 rle with new method : 0.8941586017608643 create new chi : 2.3876335620880127 time to delete rle : 0.0014646053314208984 batch 1 Loaded 2 chid ids of type : 3726 Number RLEs to save : 12260 TO DO : save crop sub photo not yet done ! save time : 0.7792792320251465 map_output_result : {1385913856: (0.0, 'Should be the crop_list due to order', 0.0), 1385913854: (0.0, 'Should be the crop_list due to order', 0.0), 1385913850: (0.0, 'Should be the crop_list due to order', 0.0), 1385913845: (0.0, 'Should be the crop_list due to order', 0.0), 1385913842: (0.0, 'Should be the crop_list due to order', 0.0), 1385913835: (0.0, 'Should be the crop_list due to order', 0.0), 1385913575: (0.0, 'Should be the crop_list due to order', 0.0), 1385913570: (0.0, 'Should be the crop_list due to order', 0.0), 1385913567: (0.0, 'Should be the crop_list due to order', 0.0), 1385913565: (0.0, 'Should be the crop_list due to order', 0.0), 1385913562: (0.0, 'Should be the crop_list due to order', 0.0), 1385913560: (0.0, 'Should be the crop_list due to order', 0.0), 1385913240: (0.0, 'Should be the crop_list due to order', 0.0), 1385913239: (0.0, 'Should be the crop_list due to order', 0.0), 1385913238: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1385913856, 1385913854, 1385913850, 1385913845, 1385913842, 1385913835, 1385913575, 1385913570, 1385913567, 1385913565, 1385913562, 1385913560, 1385913240, 1385913239, 1385913238] Looping around the photos to save general results len do output : 15 /1385913856.Didn't retrieve data . /1385913854.Didn't retrieve data . /1385913850.Didn't retrieve data . /1385913845.Didn't retrieve data . /1385913842.Didn't retrieve data . /1385913835.Didn't retrieve data . /1385913575.Didn't retrieve data . /1385913570.Didn't retrieve data . /1385913567.Didn't retrieve data . /1385913565.Didn't retrieve data . /1385913562.Didn't retrieve data . /1385913560.Didn't retrieve data . /1385913240.Didn't retrieve data . /1385913239.Didn't retrieve data . /1385913238.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913856', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913854', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913850', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913845', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913842', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913835', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913575', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913570', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913567', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913565', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913562', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913560', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913240', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913239', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913238', None, None, None, None, None, '3772637') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 45 time used for this insertion : 0.017116785049438477 save_final save missing photos in datou_result : time spend for datou_step_exec : 247.68163084983826 time spend to save output : 0.05402112007141113 total time spend for step 8 : 247.73565196990967 step9:crop_condition Fri Sep 26 11:38:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 15 ! batch 1 Loaded 108 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 4869462 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758879521_66244 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 6 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.5249435901641846 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758879534_66244 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.5218517780303955 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758879542_66244 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.8979017734527588 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! About to upload 7 photos upload in portfolio : 4869462 init cache_photo without model_param we have 7 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758879555_66244 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 7 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.3757996559143066 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! About to upload 8 photos upload in portfolio : 4869462 init cache_photo without model_param we have 8 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758879565_66244 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 8 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.9300010204315186 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758879577_66244 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.5733847618103027 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1385913856, 1385913854, 1385913850, 1385913845, 1385913842, 1385913835, 1385913575, 1385913570, 1385913567, 1385913565, 1385913562, 1385913560, 1385913240, 1385913239, 1385913238] Looping around the photos to save general results len do output : 28 /1386334945Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386334946Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386334948Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386334949Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386334950Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386334952Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386334985Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386334991Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335001Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335079Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335080Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335081Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335083Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335084Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335085Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335087Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335159Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335161Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335162Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335164Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335165Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335166Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335168Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335169Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335226Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335228Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335230Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335232Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913856', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913854', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913850', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913845', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913842', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913835', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913575', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913570', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913567', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913565', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913562', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913560', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913240', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913239', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913238', None, None, None, None, None, '3772637') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 99 time used for this insertion : 0.0455324649810791 save_final save missing photos in datou_result : time spend for datou_step_exec : 71.401132106781 time spend to save output : 0.04791426658630371 total time spend for step 9 : 71.44904637336731 step10:ventilate_hashtags_in_portfolio Fri Sep 26 11:39:40 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 27193433 get user id for portfolio 27193433 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27193433 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','flou','metal','Carton_gris','Carton_brun','Teint_Dans_La_Masse','environnement','papier','cartonnette','kraft','plastique','autre_refus')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27193433 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','flou','metal','Carton_gris','Carton_brun','Teint_Dans_La_Masse','environnement','papier','cartonnette','kraft','plastique','autre_refus')) AND mptpi.`min_score`=0.5 To do To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27193433 AND mptpi.`type`=4417 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','flou','metal','Carton_gris','Carton_brun','Teint_Dans_La_Masse','environnement','papier','cartonnette','kraft','plastique','autre_refus')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27193433 AND mptpi.`type`=4417 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','flou','metal','Carton_gris','Carton_brun','Teint_Dans_La_Masse','environnement','papier','cartonnette','kraft','plastique','autre_refus')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://marlene.fotonower.com/velours/27273372,27273373,27273374,27273375,27273376,27273377,27273378,27273379,27273380,27273381,27273382,27273383?tags=mal_croppe,flou,metal,Carton_gris,Carton_brun,Teint_Dans_La_Masse,environnement,papier,cartonnette,kraft,plastique,autre_refus Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1385913856, 1385913854, 1385913850, 1385913845, 1385913842, 1385913835, 1385913575, 1385913570, 1385913567, 1385913565, 1385913562, 1385913560, 1385913240, 1385913239, 1385913238] Looping around the photos to save general results len do output : 1 /27193433. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913856', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913854', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913850', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913845', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913842', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913835', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913575', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913570', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913567', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913565', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913562', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913560', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913240', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913239', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913238', None, None, None, None, None, '3772637') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 16 time used for this insertion : 0.01748490333557129 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.095947265625 time spend to save output : 0.01813340187072754 total time spend for step 10 : 1.1140806674957275 step11:final Fri Sep 26 11:39:41 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1385913856: ('0.0994719176427741',), 1385913854: ('0.0994719176427741',), 1385913850: ('0.0994719176427741',), 1385913845: ('0.0994719176427741',), 1385913842: ('0.0994719176427741',), 1385913835: ('0.0994719176427741',), 1385913575: ('0.0994719176427741',), 1385913570: ('0.0994719176427741',), 1385913567: ('0.0994719176427741',), 1385913565: ('0.0994719176427741',), 1385913562: ('0.0994719176427741',), 1385913560: ('0.0994719176427741',), 1385913240: ('0.0994719176427741',), 1385913239: ('0.0994719176427741',), 1385913238: ('0.0994719176427741',)} new output for save of step final : {1385913856: ('0.0994719176427741',), 1385913854: ('0.0994719176427741',), 1385913850: ('0.0994719176427741',), 1385913845: ('0.0994719176427741',), 1385913842: ('0.0994719176427741',), 1385913835: ('0.0994719176427741',), 1385913575: ('0.0994719176427741',), 1385913570: ('0.0994719176427741',), 1385913567: ('0.0994719176427741',), 1385913565: ('0.0994719176427741',), 1385913562: ('0.0994719176427741',), 1385913560: ('0.0994719176427741',), 1385913240: ('0.0994719176427741',), 1385913239: ('0.0994719176427741',), 1385913238: ('0.0994719176427741',)} [1385913856, 1385913854, 1385913850, 1385913845, 1385913842, 1385913835, 1385913575, 1385913570, 1385913567, 1385913565, 1385913562, 1385913560, 1385913240, 1385913239, 1385913238] Looping around the photos to save general results len do output : 15 /1385913856.Didn't retrieve data . /1385913854.Didn't retrieve data . /1385913850.Didn't retrieve data . /1385913845.Didn't retrieve data . /1385913842.Didn't retrieve data . /1385913835.Didn't retrieve data . /1385913575.Didn't retrieve data . /1385913570.Didn't retrieve data . /1385913567.Didn't retrieve data . /1385913565.Didn't retrieve data . /1385913562.Didn't retrieve data . /1385913560.Didn't retrieve data . /1385913240.Didn't retrieve data . /1385913239.Didn't retrieve data . /1385913238.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913856', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913854', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913850', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913845', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913842', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913835', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913575', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913570', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913567', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913565', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913562', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913560', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913240', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913239', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913238', None, None, None, None, None, '3772637') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 45 time used for this insertion : 0.026151418685913086 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.14678049087524414 time spend to save output : 0.027521133422851562 total time spend for step 11 : 0.1743016242980957 step12:velours_tree Fri Sep 26 11:39:41 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.41679978370666504 time spend to save output : 6.699562072753906e-05 total time spend for step 12 : 0.4168667793273926 step13:send_mail_cod Fri Sep 26 11:39:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin in order to get the selector url, please entre the license of selector results_Auto_P27193433_26-09-2025_11_39_42.pdf 27273359 imagette272733591758879582 27273360 imagette272733601758879582 27273361 imagette272733611758879582 27273367 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272733671758879582 27273368 imagette272733681758879582 27273369 change filename to text .change filename to text .change filename to text .change filename to text .imagette272733691758879582 27273370 change filename to text .change filename to text .imagette272733701758879583 27273362 change filename to text .imagette272733621758879583 27273363 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272733631758879583 27273364 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272733641758879584 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=27193433 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://marlene.fotonower.com/velours/27273372,27273373,27273374,27273375,27273376,27273377,27273378,27273379,27273380,27273381,27273382,27273383?tags=mal_croppe,flou,metal,Carton_gris,Carton_brun,Teint_Dans_La_Masse,environnement,papier,cartonnette,kraft,plastique,autre_refus your option no_mail is active, we will not send the real mail to your client args[1385913856] : ((1385913856, 0.08534039615003322, 2107752395), (1385913856, 1.1415852410922487, 492688767), '0.0994719176427741') We are sending mail with results at report@fotonower.com args[1385913854] : ((1385913854, 0.03276408265911468, 2107752395), (1385913854, 0.4073508800317015, 492688767), '0.0994719176427741') We are sending mail with results at report@fotonower.com args[1385913850] : ((1385913850, 0.050510565779015866, 2107752395), (1385913850, 0.3027355961001093, 492688767), '0.0994719176427741') We are sending mail with results at report@fotonower.com args[1385913845] : ((1385913845, 0.2073137784795005, 2107752395), (1385913845, -2.6777748397735297, 492609224), '0.0994719176427741') We are sending mail with results at report@fotonower.com args[1385913842] : ((1385913842, 0.09641946712308351, 2107752395), (1385913842, 0.14245055527483566, 492688767), '0.0994719176427741') We are sending mail with results at report@fotonower.com args[1385913835] : ((1385913835, -0.06267483748752264, 2107752395), (1385913835, -2.1434955533116296, 492609224), '0.0994719176427741') We are sending mail with results at report@fotonower.com args[1385913575] : ((1385913575, 0.08542660453161646, 2107752395), (1385913575, 0.23151719871967572, 492688767), '0.0994719176427741') We are sending mail with results at report@fotonower.com args[1385913570] : ((1385913570, -0.10152624447581218, 496442774), (1385913570, -0.048049369560924204, 492688767), '0.0994719176427741') We are sending mail with results at report@fotonower.com args[1385913567] : ((1385913567, -0.04756049066895759, 2107752395), (1385913567, 0.21383111532912, 492688767), '0.0994719176427741') We are sending mail with results at report@fotonower.com args[1385913565] : ((1385913565, -0.10340405794541159, 496442774), (1385913565, 0.28614354318155444, 492688767), '0.0994719176427741') We are sending mail with results at report@fotonower.com args[1385913562] : ((1385913562, 0.03613042186210244, 2107752395), (1385913562, 0.23645520213598398, 492688767), '0.0994719176427741') We are sending mail with results at report@fotonower.com args[1385913560] : ((1385913560, -0.68059755998225, 501862349), (1385913560, -0.14602185656991726, 492688767), '0.0994719176427741') We are sending mail with results at report@fotonower.com args[1385913240] : ((1385913240, -0.4312067440006728, 496442774), (1385913240, -0.14294574056802786, 492688767), '0.0994719176427741') We are sending mail with results at report@fotonower.com args[1385913239] : ((1385913239, -1.6238577557204605, 501862349), (1385913239, -1.561070025643792, 492688767), '0.0994719176427741') We are sending mail with results at report@fotonower.com args[1385913238] : ((1385913238, -0.092772392180863, 496442774), (1385913238, -1.5392815105878341, 492688767), '0.0994719176427741') We are sending mail with results at report@fotonower.com refus_total : 0.0994719176427741 2022-04-13 10:29:59 0 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27193433_26-09-2025_11_39_42.pdf results_Auto_P27193433_26-09-2025_11_39_42.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27193433_26-09-2025_11_39_42.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4433','27193433','results_Auto_P27193433_26-09-2025_11_39_42.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27193433_26-09-2025_11_39_42.pdf','pdf','','0.98','0.0994719176427741') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1385913856, 1385913854, 1385913850, 1385913845, 1385913842, 1385913835, 1385913575, 1385913570, 1385913567, 1385913565, 1385913562, 1385913560, 1385913240, 1385913239, 1385913238] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913856', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913854', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913850', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913845', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913842', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913835', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913575', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913570', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913567', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913565', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913562', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913560', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913240', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913239', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913238', None, None, None, None, None, '3772637') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 15 time used for this insertion : 0.01350712776184082 save_final save missing photos in datou_result : time spend for datou_step_exec : 7.914397716522217 time spend to save output : 0.013881921768188477 total time spend for step 13 : 7.928279638290405 step14:split_time_score Fri Sep 26 11:39:49 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('10', 85),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 24092025 27193433 Nombre de photos uploadées : 85 / 23040 (0%) 24092025 27193433 Nombre de photos taguées (types de déchets): 0 / 85 (0%) 24092025 27193433 Nombre de photos taguées (volume) : 0 / 85 (0%) elapsed_time : load_data_split_time_score 1.6689300537109375e-06 elapsed_time : order_list_meta_photo_and_scores 5.245208740234375e-06 ????????????????????????????????????????????????????????????????????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.005135297775268555 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.21685028076171875 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27193431 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27193432 order by id desc limit 1 Qualite : 0.0994719176427741 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27193433_26-09-2025_11_39_42.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27193433 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27193433 AND mptpi.`type`=3726 To do Qualite : 0.11091927628898955 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27193435 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27193435 AND mptpi.`type`=3726 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'24092025': {'nb_upload': 85, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1385913856, 1385913854, 1385913850, 1385913845, 1385913842, 1385913835, 1385913575, 1385913570, 1385913567, 1385913565, 1385913562, 1385913560, 1385913240, 1385913239, 1385913238] Looping around the photos to save general results len do output : 1 /27193433Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913856', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913854', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913850', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913845', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913842', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913835', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913575', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913570', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913567', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913565', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913562', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913560', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913240', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913239', None, None, None, None, None, '3772637') ('4433', None, None, None, None, None, None, None, '3772637') ('4433', '27193433', '1385913238', None, None, None, None, None, '3772637') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 16 time used for this insertion : 0.01433873176574707 save_final save missing photos in datou_result : time spend for datou_step_exec : 3.379396915435791 time spend to save output : 0.014616250991821289 total time spend for step 14 : 3.3940131664276123 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 347.23user 325.37system 12:59.96elapsed 86%CPU (0avgtext+0avgdata 4560240maxresident)k 10091440inputs+351992outputs (385179major+29502542minor)pagefaults 0swaps