python /home/admin/mtr/script_for_cron.py -j default -m 20 -a 'python3 ~/workarea/git/Velours/python/prod/datou.py -j batch_current -a 4433' -s traitement_sts -M 0 -S 0 -U 100,80,95 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/home/admin/workarea/git/apy', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 42667 load datou : 4433 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4433, datou_cur_ids : ['3776893'] with mtr_portfolio_ids : ['27219530'] and first list_photo_ids : [] new path : /proc/42667/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, brightness, blur_detection, crop_condition, thcl, tfhub_classification2, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 15 list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 15 ; length of list_pids : 15 ; length of list_args : 15 time to download the photos : 4.880491733551025 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 14 step1:mask_detect Fri Sep 26 11:08:22 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3021 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-09-26 11:08:27.041863: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-26 11:08:27.084519: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-26 11:08:27.087507: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f7724000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-26 11:08:27.087584: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-26 11:08:27.093977: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-26 11:08:27.285304: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x28003140 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-26 11:08:27.285374: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-26 11:08:27.286975: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:08:27.288862: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:08:27.322306: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:08:27.343665: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:08:27.348353: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:08:27.382820: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:08:27.388051: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:08:27.447668: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:08:27.449003: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:08:27.449503: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:08:27.451192: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-26 11:08:27.451214: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-26 11:08:27.451225: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-26 11:08:27.452697: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2569 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-09-26 11:08:27.945445: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:08:27.945577: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:08:27.945600: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:08:27.945617: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:08:27.945634: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:08:27.945650: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:08:27.945664: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:08:27.945680: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:08:27.946561: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:08:27.947801: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:08:27.947875: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:08:27.947893: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:08:27.947908: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:08:27.947923: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:08:27.947939: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:08:27.947954: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:08:27.947969: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:08:27.948836: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:08:27.948903: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-26 11:08:27.948914: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-26 11:08:27.948923: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-26 11:08:27.949841: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2569 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-09-26 11:08:44.139434: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:08:44.606644: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:08:47.075061: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:08:47.075585: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:08:47.082146: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:08:47.082171: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:08:47.134121: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:08:47.134201: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:08:47.175068: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.09GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:08:47.175122: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.09GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:08:47.224086: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.15GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:08:47.224124: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.15GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:08:47.226716: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.51G (1621688320 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.227278: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.36G (1459519488 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.565093: W tensorflow/core/common_runtime/bfc_allocator.cc:311] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature. 2025-09-26 11:08:47.611074: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.612217: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.613744: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.614842: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.625822: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.626383: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.626948: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.627508: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.628061: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.628632: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.645265: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.645839: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.767386: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.768001: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.775918: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.776510: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.812750: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.813534: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.814166: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.814766: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.818883: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.819493: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.820051: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.820619: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.823772: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.823796: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-09-26 11:08:47.832114: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.832939: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.842460: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.843057: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.843641: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.844208: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.844792: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:08:47.845356: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 15 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 5.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 5.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 8.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 2.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 16 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 11 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 5 Detection mask done ! Trying to reset tf kernel 42883 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1828 tf kernel not reseted sub process len(results) : 15 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 15 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 3021 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.04854536056518555 nb_pixel_total : 513624 time to create 1 rle with new method : 0.04546070098876953 length of segment : 703 time for calcul the mask position with numpy : 0.0003914833068847656 nb_pixel_total : 19485 time to create 1 rle with old method : 0.02297520637512207 length of segment : 130 time for calcul the mask position with numpy : 0.05037999153137207 nb_pixel_total : 938398 time to create 1 rle with new method : 0.3690757751464844 length of segment : 2281 time for calcul the mask position with numpy : 0.003037691116333008 nb_pixel_total : 206037 time to create 1 rle with new method : 0.007790803909301758 length of segment : 677 time for calcul the mask position with numpy : 0.008655071258544922 nb_pixel_total : 609533 time to create 1 rle with new method : 0.04790329933166504 length of segment : 875 time for calcul the mask position with numpy : 0.0006139278411865234 nb_pixel_total : 40863 time to create 1 rle with old method : 0.04953598976135254 length of segment : 240 time for calcul the mask position with numpy : 0.007447004318237305 nb_pixel_total : 548286 time to create 1 rle with new method : 0.06791234016418457 length of segment : 697 time for calcul the mask position with numpy : 0.010330438613891602 nb_pixel_total : 713909 time to create 1 rle with new method : 0.02785205841064453 length of segment : 775 time for calcul the mask position with numpy : 0.0012049674987792969 nb_pixel_total : 81837 time to create 1 rle with old method : 0.10227131843566895 length of segment : 340 time for calcul the mask position with numpy : 0.0006003379821777344 nb_pixel_total : 36762 time to create 1 rle with old method : 0.0420222282409668 length of segment : 324 time for calcul the mask position with numpy : 0.0005543231964111328 nb_pixel_total : 36423 time to create 1 rle with old method : 0.042726993560791016 length of segment : 205 time for calcul the mask position with numpy : 0.0012700557708740234 nb_pixel_total : 84119 time to create 1 rle with old method : 0.10314345359802246 length of segment : 369 time for calcul the mask position with numpy : 0.0004742145538330078 nb_pixel_total : 21424 time to create 1 rle with old method : 0.025074005126953125 length of segment : 171 time for calcul the mask position with numpy : 0.010976791381835938 nb_pixel_total : 591819 time to create 1 rle with new method : 0.05033993721008301 length of segment : 1407 time for calcul the mask position with numpy : 0.0019180774688720703 nb_pixel_total : 109180 time to create 1 rle with old method : 0.12450623512268066 length of segment : 462 time for calcul the mask position with numpy : 0.0008072853088378906 nb_pixel_total : 49254 time to create 1 rle with old method : 0.05604743957519531 length of segment : 327 time for calcul the mask position with numpy : 0.0003867149353027344 nb_pixel_total : 16781 time to create 1 rle with old method : 0.01973128318786621 length of segment : 186 time for calcul the mask position with numpy : 0.00502467155456543 nb_pixel_total : 320265 time to create 1 rle with new method : 0.013408184051513672 length of segment : 945 time for calcul the mask position with numpy : 0.0007927417755126953 nb_pixel_total : 52715 time to create 1 rle with old method : 0.06031060218811035 length of segment : 327 time for calcul the mask position with numpy : 0.004884958267211914 nb_pixel_total : 290608 time to create 1 rle with new method : 0.011488676071166992 length of segment : 1085 time for calcul the mask position with numpy : 0.0007793903350830078 nb_pixel_total : 51027 time to create 1 rle with old method : 0.05822014808654785 length of segment : 308 time for calcul the mask position with numpy : 0.035511016845703125 nb_pixel_total : 1498082 time to create 1 rle with new method : 0.7399168014526367 length of segment : 2447 time for calcul the mask position with numpy : 0.006260871887207031 nb_pixel_total : 513575 time to create 1 rle with new method : 0.011720418930053711 length of segment : 703 time for calcul the mask position with numpy : 0.0003745555877685547 nb_pixel_total : 19484 time to create 1 rle with old method : 0.02223968505859375 length of segment : 130 time for calcul the mask position with numpy : 0.018146991729736328 nb_pixel_total : 938474 time to create 1 rle with new method : 0.11591863632202148 length of segment : 2285 time for calcul the mask position with numpy : 0.0027620792388916016 nb_pixel_total : 206031 time to create 1 rle with new method : 0.007222414016723633 length of segment : 677 time for calcul the mask position with numpy : 0.00801229476928711 nb_pixel_total : 609474 time to create 1 rle with new method : 0.031225204467773438 length of segment : 874 time for calcul the mask position with numpy : 0.0006108283996582031 nb_pixel_total : 40866 time to create 1 rle with old method : 0.04513669013977051 length of segment : 240 time for calcul the mask position with numpy : 0.006937742233276367 nb_pixel_total : 548289 time to create 1 rle with new method : 0.03276658058166504 length of segment : 697 time for calcul the mask position with numpy : 0.009822368621826172 nb_pixel_total : 713935 time to create 1 rle with new method : 0.037924766540527344 length of segment : 775 time for calcul the mask position with numpy : 0.0011477470397949219 nb_pixel_total : 81819 time to create 1 rle with old method : 0.09188985824584961 length of segment : 340 time for calcul the mask position with numpy : 0.0009329319000244141 nb_pixel_total : 36763 time to create 1 rle with old method : 0.06171417236328125 length of segment : 324 time for calcul the mask position with numpy : 0.0006961822509765625 nb_pixel_total : 36430 time to create 1 rle with old method : 0.04547286033630371 length of segment : 205 time for calcul the mask position with numpy : 0.0012714862823486328 nb_pixel_total : 84095 time to create 1 rle with old method : 0.09824848175048828 length of segment : 369 time for calcul the mask position with numpy : 0.0006203651428222656 nb_pixel_total : 49662 time to create 1 rle with old method : 0.05389738082885742 length of segment : 207 time for calcul the mask position with numpy : 0.0005881786346435547 nb_pixel_total : 29007 time to create 1 rle with old method : 0.03247380256652832 length of segment : 221 time for calcul the mask position with numpy : 0.0007729530334472656 nb_pixel_total : 44141 time to create 1 rle with old method : 0.04833173751831055 length of segment : 298 time for calcul the mask position with numpy : 0.0027451515197753906 nb_pixel_total : 162069 time to create 1 rle with new method : 0.00994253158569336 length of segment : 587 time for calcul the mask position with numpy : 0.00043463706970214844 nb_pixel_total : 33170 time to create 1 rle with old method : 0.036809444427490234 length of segment : 145 time for calcul the mask position with numpy : 0.05989551544189453 nb_pixel_total : 1470265 time to create 1 rle with new method : 0.24663352966308594 length of segment : 2635 time for calcul the mask position with numpy : 0.0013797283172607422 nb_pixel_total : 106264 time to create 1 rle with old method : 0.1132974624633789 length of segment : 276 time for calcul the mask position with numpy : 0.002674579620361328 nb_pixel_total : 223792 time to create 1 rle with new method : 0.005688905715942383 length of segment : 595 time for calcul the mask position with numpy : 0.001253366470336914 nb_pixel_total : 95575 time to create 1 rle with old method : 0.10206079483032227 length of segment : 296 time for calcul the mask position with numpy : 0.0003287792205810547 nb_pixel_total : 24624 time to create 1 rle with old method : 0.02656412124633789 length of segment : 193 time for calcul the mask position with numpy : 0.0014390945434570312 nb_pixel_total : 94831 time to create 1 rle with old method : 0.10434579849243164 length of segment : 339 time for calcul the mask position with numpy : 0.002478361129760742 nb_pixel_total : 176932 time to create 1 rle with new method : 0.00564265251159668 length of segment : 812 time for calcul the mask position with numpy : 0.02941608428955078 nb_pixel_total : 1274122 time to create 1 rle with new method : 0.2999999523162842 length of segment : 3012 time for calcul the mask position with numpy : 0.04152941703796387 nb_pixel_total : 1834552 time to create 1 rle with new method : 0.42929697036743164 length of segment : 2654 time for calcul the mask position with numpy : 0.0014805793762207031 nb_pixel_total : 52571 time to create 1 rle with old method : 0.058423757553100586 length of segment : 349 time for calcul the mask position with numpy : 0.0013549327850341797 nb_pixel_total : 24171 time to create 1 rle with old method : 0.028989076614379883 length of segment : 311 time for calcul the mask position with numpy : 0.0018160343170166016 nb_pixel_total : 31288 time to create 1 rle with old method : 0.035920143127441406 length of segment : 361 time for calcul the mask position with numpy : 0.0025491714477539062 nb_pixel_total : 176820 time to create 1 rle with new method : 0.005645036697387695 length of segment : 582 time for calcul the mask position with numpy : 0.00014328956604003906 nb_pixel_total : 7115 time to create 1 rle with old method : 0.008528709411621094 length of segment : 83 time for calcul the mask position with numpy : 0.006592988967895508 nb_pixel_total : 526792 time to create 1 rle with new method : 0.01265406608581543 length of segment : 660 time for calcul the mask position with numpy : 0.02048468589782715 nb_pixel_total : 1023496 time to create 1 rle with new method : 0.21237969398498535 length of segment : 2091 time for calcul the mask position with numpy : 8.940696716308594e-05 nb_pixel_total : 2017 time to create 1 rle with old method : 0.002346038818359375 length of segment : 93 time for calcul the mask position with numpy : 0.0019443035125732422 nb_pixel_total : 179869 time to create 1 rle with new method : 0.003759145736694336 length of segment : 318 time for calcul the mask position with numpy : 0.00039696693420410156 nb_pixel_total : 20401 time to create 1 rle with old method : 0.023855924606323242 length of segment : 176 time for calcul the mask position with numpy : 0.0038263797760009766 nb_pixel_total : 178841 time to create 1 rle with new method : 0.00908660888671875 length of segment : 610 time for calcul the mask position with numpy : 0.0051534175872802734 nb_pixel_total : 318156 time to create 1 rle with new method : 0.008882999420166016 length of segment : 527 time for calcul the mask position with numpy : 0.0024099349975585938 nb_pixel_total : 119373 time to create 1 rle with old method : 0.1319875717163086 length of segment : 497 time for calcul the mask position with numpy : 0.002445220947265625 nb_pixel_total : 98412 time to create 1 rle with old method : 0.11780977249145508 length of segment : 548 time for calcul the mask position with numpy : 0.0013477802276611328 nb_pixel_total : 54962 time to create 1 rle with old method : 0.06050372123718262 length of segment : 473 time for calcul the mask position with numpy : 0.0038330554962158203 nb_pixel_total : 185443 time to create 1 rle with new method : 0.006463527679443359 length of segment : 816 time for calcul the mask position with numpy : 0.0008304119110107422 nb_pixel_total : 31252 time to create 1 rle with old method : 0.03460550308227539 length of segment : 212 time for calcul the mask position with numpy : 0.000274658203125 nb_pixel_total : 8026 time to create 1 rle with old method : 0.009176492691040039 length of segment : 104 time for calcul the mask position with numpy : 0.007767200469970703 nb_pixel_total : 436250 time to create 1 rle with new method : 0.010349750518798828 length of segment : 696 time for calcul the mask position with numpy : 0.0013761520385742188 nb_pixel_total : 39305 time to create 1 rle with old method : 0.04446053504943848 length of segment : 358 time for calcul the mask position with numpy : 0.003991603851318359 nb_pixel_total : 158066 time to create 1 rle with new method : 0.006743669509887695 length of segment : 539 time for calcul the mask position with numpy : 0.0011725425720214844 nb_pixel_total : 19998 time to create 1 rle with old method : 0.022737741470336914 length of segment : 321 time for calcul the mask position with numpy : 0.00602412223815918 nb_pixel_total : 215462 time to create 1 rle with new method : 0.011549234390258789 length of segment : 773 time for calcul the mask position with numpy : 0.0009195804595947266 nb_pixel_total : 45273 time to create 1 rle with old method : 0.051782846450805664 length of segment : 378 time for calcul the mask position with numpy : 0.002622842788696289 nb_pixel_total : 147277 time to create 1 rle with old method : 0.18799138069152832 length of segment : 366 time for calcul the mask position with numpy : 0.00037550926208496094 nb_pixel_total : 19309 time to create 1 rle with old method : 0.02158069610595703 length of segment : 180 time for calcul the mask position with numpy : 0.0016605854034423828 nb_pixel_total : 65887 time to create 1 rle with old method : 0.0741419792175293 length of segment : 302 time for calcul the mask position with numpy : 0.00014710426330566406 nb_pixel_total : 2354 time to create 1 rle with old method : 0.002869844436645508 length of segment : 57 time for calcul the mask position with numpy : 0.0007052421569824219 nb_pixel_total : 23381 time to create 1 rle with old method : 0.026468992233276367 length of segment : 241 time for calcul the mask position with numpy : 0.0008661746978759766 nb_pixel_total : 30971 time to create 1 rle with old method : 0.03483104705810547 length of segment : 204 time for calcul the mask position with numpy : 0.0018923282623291016 nb_pixel_total : 61666 time to create 1 rle with old method : 0.06936407089233398 length of segment : 372 time for calcul the mask position with numpy : 0.005153656005859375 nb_pixel_total : 237365 time to create 1 rle with new method : 0.011045217514038086 length of segment : 489 time for calcul the mask position with numpy : 0.015468358993530273 nb_pixel_total : 710387 time to create 1 rle with new method : 0.17615890502929688 length of segment : 1167 time for calcul the mask position with numpy : 0.0012328624725341797 nb_pixel_total : 59930 time to create 1 rle with old method : 0.06647038459777832 length of segment : 341 time for calcul the mask position with numpy : 0.001068115234375 nb_pixel_total : 21991 time to create 1 rle with old method : 0.024745702743530273 length of segment : 281 time for calcul the mask position with numpy : 0.0031423568725585938 nb_pixel_total : 93605 time to create 1 rle with old method : 0.10474109649658203 length of segment : 371 time for calcul the mask position with numpy : 0.0048046112060546875 nb_pixel_total : 159397 time to create 1 rle with new method : 0.007657051086425781 length of segment : 736 time for calcul the mask position with numpy : 0.0002675056457519531 nb_pixel_total : 5617 time to create 1 rle with old method : 0.006288051605224609 length of segment : 141 time for calcul the mask position with numpy : 0.0005660057067871094 nb_pixel_total : 29725 time to create 1 rle with old method : 0.034155845642089844 length of segment : 126 time for calcul the mask position with numpy : 0.0016131401062011719 nb_pixel_total : 71201 time to create 1 rle with old method : 0.08095836639404297 length of segment : 366 time for calcul the mask position with numpy : 0.00042748451232910156 nb_pixel_total : 9522 time to create 1 rle with old method : 0.010947942733764648 length of segment : 171 time for calcul the mask position with numpy : 0.0009379386901855469 nb_pixel_total : 27546 time to create 1 rle with old method : 0.03072500228881836 length of segment : 260 time for calcul the mask position with numpy : 0.0020227432250976562 nb_pixel_total : 92424 time to create 1 rle with old method : 0.10548853874206543 length of segment : 537 time for calcul the mask position with numpy : 0.0005931854248046875 nb_pixel_total : 20032 time to create 1 rle with old method : 0.022208452224731445 length of segment : 171 time for calcul the mask position with numpy : 0.0017173290252685547 nb_pixel_total : 71481 time to create 1 rle with old method : 0.08032083511352539 length of segment : 363 time for calcul the mask position with numpy : 0.0019741058349609375 nb_pixel_total : 80485 time to create 1 rle with old method : 0.08893680572509766 length of segment : 304 time for calcul the mask position with numpy : 0.03194379806518555 nb_pixel_total : 1457198 time to create 1 rle with new method : 0.29396700859069824 length of segment : 2559 time for calcul the mask position with numpy : 0.002153158187866211 nb_pixel_total : 116339 time to create 1 rle with old method : 0.12576651573181152 length of segment : 505 time for calcul the mask position with numpy : 0.001943826675415039 nb_pixel_total : 82964 time to create 1 rle with old method : 0.0913228988647461 length of segment : 242 time for calcul the mask position with numpy : 0.0010256767272949219 nb_pixel_total : 62422 time to create 1 rle with old method : 0.07027101516723633 length of segment : 337 time for calcul the mask position with numpy : 0.0033457279205322266 nb_pixel_total : 112484 time to create 1 rle with old method : 0.12636446952819824 length of segment : 535 time for calcul the mask position with numpy : 0.0002810955047607422 nb_pixel_total : 11110 time to create 1 rle with old method : 0.013032913208007812 length of segment : 164 time for calcul the mask position with numpy : 0.0006282329559326172 nb_pixel_total : 26445 time to create 1 rle with old method : 0.030788183212280273 length of segment : 151 time for calcul the mask position with numpy : 0.00016999244689941406 nb_pixel_total : 2644 time to create 1 rle with old method : 0.003587007522583008 length of segment : 24 time for calcul the mask position with numpy : 0.001970529556274414 nb_pixel_total : 128401 time to create 1 rle with old method : 0.15066313743591309 length of segment : 581 time for calcul the mask position with numpy : 0.010309696197509766 nb_pixel_total : 510887 time to create 1 rle with new method : 0.01415109634399414 length of segment : 775 time for calcul the mask position with numpy : 0.04732465744018555 nb_pixel_total : 526699 time to create 1 rle with new method : 0.6848955154418945 length of segment : 1463 time for calcul the mask position with numpy : 0.0012052059173583984 nb_pixel_total : 77154 time to create 1 rle with old method : 0.08654356002807617 length of segment : 317 time for calcul the mask position with numpy : 0.005880594253540039 nb_pixel_total : 255536 time to create 1 rle with new method : 0.009411811828613281 length of segment : 675 time for calcul the mask position with numpy : 0.0028629302978515625 nb_pixel_total : 56183 time to create 1 rle with old method : 0.06370759010314941 length of segment : 525 time for calcul the mask position with numpy : 0.0021028518676757812 nb_pixel_total : 101778 time to create 1 rle with old method : 0.1149296760559082 length of segment : 215 time for calcul the mask position with numpy : 0.0002193450927734375 nb_pixel_total : 3336 time to create 1 rle with old method : 0.004194021224975586 length of segment : 41 time for calcul the mask position with numpy : 0.030940532684326172 nb_pixel_total : 954642 time to create 1 rle with new method : 0.23080110549926758 length of segment : 2228 time for calcul the mask position with numpy : 0.0007157325744628906 nb_pixel_total : 18935 time to create 1 rle with old method : 0.0219271183013916 length of segment : 117 time for calcul the mask position with numpy : 0.006136894226074219 nb_pixel_total : 319483 time to create 1 rle with new method : 0.009279727935791016 length of segment : 531 time for calcul the mask position with numpy : 0.0007534027099609375 nb_pixel_total : 26565 time to create 1 rle with old method : 0.03843402862548828 length of segment : 220 time for calcul the mask position with numpy : 0.047759056091308594 nb_pixel_total : 727034 time to create 1 rle with new method : 0.24658608436584473 length of segment : 2800 time for calcul the mask position with numpy : 0.004267454147338867 nb_pixel_total : 167347 time to create 1 rle with new method : 0.013251066207885742 length of segment : 1029 time for calcul the mask position with numpy : 0.0037033557891845703 nb_pixel_total : 231130 time to create 1 rle with new method : 0.00666046142578125 length of segment : 424 time for calcul the mask position with numpy : 0.0004963874816894531 nb_pixel_total : 31874 time to create 1 rle with old method : 0.034780025482177734 length of segment : 222 time for calcul the mask position with numpy : 0.0003902912139892578 nb_pixel_total : 16821 time to create 1 rle with old method : 0.019685029983520508 length of segment : 146 time for calcul the mask position with numpy : 0.017702817916870117 nb_pixel_total : 914927 time to create 1 rle with new method : 0.24581265449523926 length of segment : 2587 time for calcul the mask position with numpy : 0.027075767517089844 nb_pixel_total : 225502 time to create 1 rle with new method : 0.5689058303833008 length of segment : 1264 time for calcul the mask position with numpy : 0.15716004371643066 nb_pixel_total : 3187794 time to create 1 rle with new method : 0.9625182151794434 length of segment : 5025 time for calcul the mask position with numpy : 0.0007319450378417969 nb_pixel_total : 49303 time to create 1 rle with old method : 0.05648326873779297 length of segment : 302 time for calcul the mask position with numpy : 0.004523038864135742 nb_pixel_total : 326134 time to create 1 rle with new method : 0.012776374816894531 length of segment : 665 time for calcul the mask position with numpy : 0.08335709571838379 nb_pixel_total : 2294798 time to create 1 rle with new method : 0.19165587425231934 length of segment : 3406 time for calcul the mask position with numpy : 0.0019807815551757812 nb_pixel_total : 100330 time to create 1 rle with old method : 0.11356163024902344 length of segment : 340 time spent for convertir_results : 29.90564250946045 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 128 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 85007 save missing photos in datou_result : time spend for datou_step_exec : 130.73872017860413 time spend to save output : 5.018048286437988 total time spend for step 1 : 135.75676846504211 step2:brightness Fri Sep 26 11:10:38 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness toutes les photos sont déjà traitées, on saute les calculs Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 15 time used for this insertion : 0.009784936904907227 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 15 time used for this insertion : 0.008903741836547852 save missing photos in datou_result : time spend for datou_step_exec : 0.023107290267944336 time spend to save output : 0.022924184799194336 total time spend for step 2 : 0.04603147506713867 step3:blur_detection Fri Sep 26 11:10:38 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection toutes les photos sont déjà traitées, on saute les calculs Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 15 time used for this insertion : 0.009758234024047852 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 15 time used for this insertion : 0.00885629653930664 save missing photos in datou_result : time spend for datou_step_exec : 0.02666330337524414 time spend to save output : 0.022834300994873047 total time spend for step 3 : 0.04949760437011719 step4:crop_condition Fri Sep 26 11:10:38 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 15 ! batch 1 Loaded 128 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 12 About to insert : list_path_to_insert length 12 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 28 About to insert : list_path_to_insert length 28 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 20 About to insert : list_path_to_insert length 20 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1385833634, 1385830344, 1385830342, 1385829658, 1385829652, 1385829650, 1385829644, 1385829643, 1385829641, 1385827780, 1385827744, 1385827706, 1385827651, 1385827604, 1385827571] Looping around the photos to save general results len do output : 86 /-3973260004Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260016Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260026Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260034Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260037Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260065Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260067Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262849Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262850Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262848Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262858Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262856Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260018Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260039Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260040Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260050Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260052Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262835Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262861Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262873Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262881Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260003Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260007Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260025Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260029Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260057Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260070Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262839Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262851Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260006Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973259999Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260010Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260013Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260012Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260014Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260028Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260021Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260032Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260035Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260043Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260044Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260058Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260060Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260059Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262832Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976616871Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262841Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262838Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262845Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262853Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262854Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262847Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262863Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262860Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262870Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262874Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262866Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260000Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260002Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260015Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260011Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260022Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260024Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260033Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260042Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260047Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260055Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260066Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260063Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260064Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262834Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262836Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262865Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262875Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262869Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262876Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262880Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260005Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260027Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976616870Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260041Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260062Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260069Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262840Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973262855Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3973260017Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385833634', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385830344', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385830342', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829658', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829652', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829650', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829644', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829643', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829641', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827780', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827744', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827706', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827651', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827604', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827571', None, None, None, None, None, '3776893') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 273 time used for this insertion : 0.04447817802429199 save_final save missing photos in datou_result : time spend for datou_step_exec : 100.46396613121033 time spend to save output : 0.06656694412231445 total time spend for step 4 : 100.53053307533264 step5:thcl Fri Sep 26 11:12:19 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 1 we are using the classfication for multi_thcl [2456] time to import caffe and check if the image exist : 0.015317678451538086 time to convert the images to numpy array : 0.31674671173095703 time to import caffe and check if the image exist : 0.01284933090209961 time to convert the images to numpy array : 0.31970763206481934 time to import caffe and check if the image exist : 0.014564037322998047 time to convert the images to numpy array : 0.6686387062072754 time to import caffe and check if the image exist : 0.010887384414672852 time to convert the images to numpy array : 0.6779356002807617 time to import caffe and check if the image exist : 0.01791238784790039 time to convert the images to numpy array : 0.7915377616882324 time to import caffe and check if the image exist : 0.022824525833129883 time to convert the images to numpy array : 0.7886502742767334 time to import caffe and check if the image exist : 0.014453411102294922 time to convert the images to numpy array : 0.7984905242919922 time to import caffe and check if the image exist : 0.016034364700317383 time to convert the images to numpy array : 0.9960494041442871 time to import caffe and check if the image exist : 0.017917633056640625 time to convert the images to numpy array : 1.0630147457122803 time to import caffe and check if the image exist : 0.016875028610229492 time to convert the images to numpy array : 1.0717277526855469 total time to convert the images to numpy array : 1.4475231170654297 list photo_ids error: [] list photo_ids correct : [-3973260033, -3973260042, -3973260047, -3973260055, -3973260066, -3973260063, -3973260064, -3973262834, -3973262836, -3973260062, -3973260069, -3973262840, -3973262855, -3973260017, -3973262870, -3973262874, -3973262866, -3973260000, -3973260002, -3973260015, -3973260011, -3973260022, -3973260024, -3973260004, -3973260016, -3973260026, -3973260034, -3973260037, -3973260065, -3973260067, -3973262849, -3973262850, -3976616871, -3973262841, -3973262838, -3973262845, -3973262853, -3973262854, -3973262847, -3973262863, -3973262860, -3973260021, -3973260032, -3973260035, -3973260043, -3973260044, -3973260058, -3973260060, -3973260059, -3973262832, -3973262865, -3973262875, -3973262869, -3973262876, -3973262880, -3973260005, -3973260027, -3976616870, -3973260041, -3973262848, -3973262858, -3973262856, -3973260018, -3973260039, -3973260040, -3973260050, -3973260052, -3973262835, -3973262861, -3973262873, -3973262881, -3973260003, -3973260007, -3973260025, -3973260029, -3973260057, -3973260070, -3973262839, -3973262851, -3973260006, -3973259999, -3973260010, -3973260013, -3973260012, -3973260014, -3973260028] number of photos to traite : 86 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10582 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/:/home/admin/workarea/git/apy/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10360 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.7259180545806885 time used to do the prediction : 0.2630960941314697 we don't save the descriptors for this thcl 2456 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 2.0265579223632812e-05 save missing photos in datou_result : time spend for datou_step_exec : 7.45768141746521 time spend to save output : 0.0003452301025390625 total time spend for step 5 : 7.458026647567749 step6:tfhub_classification2 Fri Sep 26 11:12:26 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou_step TFHub with tf2 ! we are using the classfication for only one thcl 3501 begin to check gpu status inside check gpu memory 2025-09-26 11:12:30.101843: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-26 11:12:30.103754: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:12:30.103822: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:12:30.103875: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:12:30.119796: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:12:30.119908: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:12:30.126236: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:12:30.128511: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:12:30.143785: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:12:30.145542: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:12:30.146690: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-26 11:12:30.180573: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-26 11:12:30.182931: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f7678000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-26 11:12:30.182974: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-26 11:12:30.189181: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x35d4b830 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-26 11:12:30.189219: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-26 11:12:30.190631: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:12:30.190709: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:12:30.190741: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:12:30.190969: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:12:30.191005: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:12:30.191055: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:12:30.191109: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:12:30.191166: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:12:30.193310: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:12:30.193708: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:12:30.193765: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-26 11:12:30.193777: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-26 11:12:30.193787: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-26 11:12:30.195399: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3096 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) l 3637 free memory gpu now : 10358 max_wait_temp : 1 max_wait : 5 1 Physical GPUs, 1 Logical GPUs tagging for thcl : 3501 To do loadFromThcl(), then load ParamDescType : thcl3501 thcls : [{'id': 3501, 'mtr_user_id': 31, 'name': 'learn_papier_mixte_centre_tf_b4_220223_3', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '9087724,9087727,9087725,9087729,9087726,9087723,9087722,9087720,9087721,9087728', 'photo_hashtag_type': 4539, 'photo_desc_type': 5759, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 3501, 'mtr_user_id': 31, 'name': 'learn_papier_mixte_centre_tf_b4_220223_3', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '9087724,9087727,9087725,9087729,9087726,9087723,9087722,9087720,9087721,9087728', 'photo_hashtag_type': 4539, 'photo_desc_type': 5759, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5759 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5759, 'learn_papier_mixte_centre_tf_b4_220223_3', 1792, 1792, 'learn_papier_mixte_centre_tf_b4_220223_3', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 2, 22, 13, 54, 32), datetime.datetime(2023, 2, 22, 13, 54, 32)) model_name : learn_papier_mixte_centre_tf_b4_220223_3 model_param file didn't exist model_name : learn_papier_mixte_centre_tf_b4_220223_3 model_type : tf_classification2 list file need : ['Confusion_Matrix.png', 'Precision_Recall_Carton_brun.jpg', 'Precision_Recall_Carton_gris.jpg', 'Precision_Recall_Teint_Dans_La_Masse.jpg', 'Precision_Recall_autre_refus.jpg', 'Precision_Recall_cartonnette.jpg', 'Precision_Recall_environnement.jpg', 'Precision_Recall_kraft.jpg', 'Precision_Recall_metal.jpg', 'Precision_Recall_papier.jpg', 'Precision_Recall_plastique.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file exist in s3 : ['Confusion_Matrix.png', 'Precision_Recall_Carton_brun.jpg', 'Precision_Recall_Carton_gris.jpg', 'Precision_Recall_Teint_Dans_La_Masse.jpg', 'Precision_Recall_autre_refus.jpg', 'Precision_Recall_cartonnette.jpg', 'Precision_Recall_environnement.jpg', 'Precision_Recall_kraft.jpg', 'Precision_Recall_metal.jpg', 'Precision_Recall_papier.jpg', 'Precision_Recall_plastique.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file manque in s3 : [] local folder : /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3 /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Confusion_Matrix.png size_local : 121269 size in s3 : 121269 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:17 Confusion_Matrix.png already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_Carton_brun.jpg size_local : 76104 size in s3 : 76104 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_Carton_brun.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_Carton_gris.jpg size_local : 82912 size in s3 : 82912 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_Carton_gris.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_Teint_Dans_La_Masse.jpg size_local : 75423 size in s3 : 75423 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:25 Precision_Recall_Teint_Dans_La_Masse.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_autre_refus.jpg size_local : 78883 size in s3 : 78883 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_autre_refus.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_cartonnette.jpg size_local : 81791 size in s3 : 81791 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:30 Precision_Recall_cartonnette.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_environnement.jpg size_local : 59936 size in s3 : 59936 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:17 Precision_Recall_environnement.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_kraft.jpg size_local : 80280 size in s3 : 80280 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:25 Precision_Recall_kraft.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_metal.jpg size_local : 77736 size in s3 : 77736 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_metal.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_papier.jpg size_local : 69041 size in s3 : 69041 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_papier.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_plastique.jpg size_local : 70770 size in s3 : 70770 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:17 Precision_Recall_plastique.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Result_Summary.txt size_local : 1092 size in s3 : 1092 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Result_Summary.txt already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/checkpoint size_local : 99 size in s3 : 99 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 checkpoint already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_checkpoint.ckpt.data-00000-of-00002 size_local : 429801 size in s3 : 429801 create time local : 2023-03-03 12:26:16 create time in s3 : 2023-02-22 12:54:30 model_checkpoint.ckpt.data-00000-of-00002 already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_checkpoint.ckpt.data-00001-of-00002 size_local : 141033188 size in s3 : 141033188 create time local : 2023-03-03 12:26:17 create time in s3 : 2023-02-22 12:54:25 model_checkpoint.ckpt.data-00001-of-00002 already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_checkpoint.ckpt.index size_local : 85618 size in s3 : 85618 create time local : 2023-03-03 12:26:17 create time in s3 : 2023-02-22 12:54:30 model_checkpoint.ckpt.index already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_weights.h5 size_local : 71289448 size in s3 : 71289448 create time local : 2023-03-03 12:26:18 create time in s3 : 2023-02-22 12:54:17 model_weights.h5 already exist and didn't need to update desc size : 1792 Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= module (KerasLayer) (None, 1792) 17673816 _________________________________________________________________ learn_papier_mixte_centre_tf (None, 10) 17930 ================================================================= Total params: 17,691,746 Trainable params: 17,930 Non-trainable params: 17,673,816 _________________________________________________________________ Loading Weights... time used to create the model : 16.507309675216675 time used to load_weights : 0.3149852752685547 0it [00:00, ?it/s] 86it [00:00, 2590.25it/s]2025-09-26 11:12:50.158219: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:12:50.164457: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 temp/1758877697_42667_1385833634_1a0f6be4ce455c7528ed152c6aa16506_rle_crop_3973260004_0.png temp/1758877697_42667_1385830344_1dc22bdc05067dfaf24b0238bd6faa2e_rle_crop_3973260016_0.png temp/1758877697_42667_1385830342_b830328de2e84db3f2c61c5897eee07b_rle_crop_3973260026_0.png temp/1758877697_42667_1385829658_805e2de710dc90af5cb441f31ed6b27e_rle_crop_3973260034_0.png temp/1758877697_42667_1385829652_4187f4d15f92a352dab5176f91bd8aaf_rle_crop_3973260037_0.png temp/1758877697_42667_1385827780_f3a931ddf32d94053cd9fb345d128c3f_rle_crop_3973260065_0.png temp/1758877697_42667_1385827780_f3a931ddf32d94053cd9fb345d128c3f_rle_crop_3973260067_0.png temp/1758877697_42667_1385827706_b90a3384d7d57091d413a90fe0bee31c_rle_crop_3973262849_0.png temp/1758877697_42667_1385827706_b90a3384d7d57091d413a90fe0bee31c_rle_crop_3973262850_0.png temp/1758877697_42667_1385827706_b90a3384d7d57091d413a90fe0bee31c_rle_crop_3973262848_0.png temp/1758877697_42667_1385827651_abaf545d3141ac0a7ea11cc1f2f8cc96_rle_crop_3973262858_0.png temp/1758877697_42667_1385827651_abaf545d3141ac0a7ea11cc1f2f8cc96_rle_crop_3973262856_0.png temp/1758877697_42667_1385830344_1dc22bdc05067dfaf24b0238bd6faa2e_rle_crop_3973260018_0.png temp/1758877697_42667_1385829652_4187f4d15f92a352dab5176f91bd8aaf_rle_crop_3973260039_0.png temp/1758877697_42667_1385829652_4187f4d15f92a352dab5176f91bd8aaf_rle_crop_3973260040_0.png temp/1758877697_42667_1385829644_48367585ff537f707912e621f70291b5_rle_crop_3973260050_0.png temp/1758877697_42667_1385829643_46d56f20a80d075defb45a70c9bb776a_rle_crop_3973260052_0.png temp/1758877697_42667_1385827744_d81372b8b770c6912718ace8b7ced5e0_rle_crop_3973262835_0.png temp/1758877697_42667_1385827651_abaf545d3141ac0a7ea11cc1f2f8cc96_rle_crop_3973262861_0.png temp/1758877697_42667_1385827604_6981a0ec05cf2e92e0e9a85ff280977b_rle_crop_3973262873_0.png temp/1758877697_42667_1385827571_bde4c433d1056e95accf6113889f54d9_rle_crop_3973262881_0.png temp/1758877697_42667_1385833634_1a0f6be4ce455c7528ed152c6aa16506_rle_crop_3973260003_0.png temp/1758877697_42667_1385833634_1a0f6be4ce455c7528ed152c6aa16506_rle_crop_3973260007_0.png temp/1758877697_42667_1385830342_b830328de2e84db3f2c61c5897eee07b_rle_crop_3973260025_0.png temp/1758877697_42667_1385830342_b830328de2e84db3f2c61c5897eee07b_rle_crop_3973260029_0.png temp/1758877697_42667_1385829643_46d56f20a80d075defb45a70c9bb776a_rle_crop_3973260057_0.png temp/1758877697_42667_1385827780_f3a931ddf32d94053cd9fb345d128c3f_rle_crop_3973260070_0.png temp/1758877697_42667_1385827744_d81372b8b770c6912718ace8b7ced5e0_rle_crop_3973262839_0.png temp/1758877697_42667_1385827706_b90a3384d7d57091d413a90fe0bee31c_rle_crop_3973262851_0.png temp/1758877697_42667_1385833634_1a0f6be4ce455c7528ed152c6aa16506_rle_crop_3973260006_0.png temp/1758877697_42667_1385833634_1a0f6be4ce455c7528ed152c6aa16506_rle_crop_3973259999_0.png temp/1758877697_42667_1385833634_1a0f6be4ce455c7528ed152c6aa16506_rle_crop_3973260010_0.png temp/1758877697_42667_1385830344_1dc22bdc05067dfaf24b0238bd6faa2e_rle_crop_3973260013_0.png temp/1758877697_42667_1385830344_1dc22bdc05067dfaf24b0238bd6faa2e_rle_crop_3973260012_0.png temp/1758877697_42667_1385830344_1dc22bdc05067dfaf24b0238bd6faa2e_rle_crop_3973260014_0.png temp/1758877697_42667_1385830342_b830328de2e84db3f2c61c5897eee07b_rle_crop_3973260028_0.png temp/1758877697_42667_1385830342_b830328de2e84db3f2c61c5897eee07b_rle_crop_3973260021_0.png temp/1758877697_42667_1385830342_b830328de2e84db3f2c61c5897eee07b_rle_crop_3973260032_0.png temp/1758877697_42667_1385829658_805e2de710dc90af5cb441f31ed6b27e_rle_crop_3973260035_0.png temp/1758877697_42667_1385829650_150a644bfebbfbdddda30686851ca49a_rle_crop_3973260043_0.png temp/1758877697_42667_1385829650_150a644bfebbfbdddda30686851ca49a_rle_crop_3973260044_0.png temp/1758877697_42667_1385829641_6aa441f240453c8550fc21046efebe7d_rle_crop_3973260058_0.png temp/1758877697_42667_1385829641_6aa441f240453c8550fc21046efebe7d_rle_crop_3973260060_0.png temp/1758877697_42667_1385829641_6aa441f240453c8550fc21046efebe7d_rle_crop_3973260059_0.png temp/1758877697_42667_1385827744_d81372b8b770c6912718ace8b7ced5e0_rle_crop_3973262832_0.png temp/1758877697_42667_1385827744_d81372b8b770c6912718ace8b7ced5e0_rle_crop_3976616871_0.png temp/1758877697_42667_1385827744_d81372b8b770c6912718ace8b7ced5e0_rle_crop_3973262841_0.png temp/1758877697_42667_1385827744_d81372b8b770c6912718ace8b7ced5e0_rle_crop_3973262838_0.png temp/1758877697_42667_1385827706_b90a3384d7d57091d413a90fe0bee31c_rle_crop_3973262845_0.png temp/1758877697_42667_1385827706_b90a3384d7d57091d413a90fe0bee31c_rle_crop_3973262853_0.png temp/1758877697_42667_1385827706_b90a3384d7d57091d413a90fe0bee31c_rle_crop_3973262854_0.png temp/1758877697_42667_1385827706_b90a3384d7d57091d413a90fe0bee31c_rle_crop_3973262847_0.png temp/1758877697_42667_1385827651_abaf545d3141ac0a7ea11cc1f2f8cc96_rle_crop_3973262863_0.png temp/1758877697_42667_1385827651_abaf545d3141ac0a7ea11cc1f2f8cc96_rle_crop_3973262860_0.png temp/1758877697_42667_1385827604_6981a0ec05cf2e92e0e9a85ff280977b_rle_crop_3973262870_0.png temp/1758877697_42667_1385827604_6981a0ec05cf2e92e0e9a85ff280977b_rle_crop_3973262874_0.png temp/1758877697_42667_1385827604_6981a0ec05cf2e92e0e9a85ff280977b_rle_crop_3973262866_0.png temp/1758877697_42667_1385833634_1a0f6be4ce455c7528ed152c6aa16506_rle_crop_3973260000_0.png temp/1758877697_42667_1385833634_1a0f6be4ce455c7528ed152c6aa16506_rle_crop_3973260002_0.png temp/1758877697_42667_1385830344_1dc22bdc05067dfaf24b0238bd6faa2e_rle_crop_3973260015_0.png temp/1758877697_42667_1385830344_1dc22bdc05067dfaf24b0238bd6faa2e_rle_crop_3973260011_0.png temp/1758877697_42667_1385830342_b830328de2e84db3f2c61c5897eee07b_rle_crop_3973260022_0.png temp/1758877697_42667_1385830342_b830328de2e84db3f2c61c5897eee07b_rle_crop_3973260024_0.png temp/1758877697_42667_1385829658_805e2de710dc90af5cb441f31ed6b27e_rle_crop_3973260033_0.png temp/1758877697_42667_1385829650_150a644bfebbfbdddda30686851ca49a_rle_crop_3973260042_0.png temp/1758877697_42667_1385829644_48367585ff537f707912e621f70291b5_rle_crop_3973260047_0.png temp/1758877697_42667_1385829643_46d56f20a80d075defb45a70c9bb776a_rle_crop_3973260055_0.png temp/1758877697_42667_1385827780_f3a931ddf32d94053cd9fb345d128c3f_rle_crop_3973260066_0.png temp/1758877697_42667_1385827780_f3a931ddf32d94053cd9fb345d128c3f_rle_crop_3973260063_0.png temp/1758877697_42667_1385827780_f3a931ddf32d94053cd9fb345d128c3f_rle_crop_3973260064_0.png temp/1758877697_42667_1385827744_d81372b8b770c6912718ace8b7ced5e0_rle_crop_3973262834_0.png temp/1758877697_42667_1385827744_d81372b8b770c6912718ace8b7ced5e0_rle_crop_3973262836_0.png temp/1758877697_42667_1385827604_6981a0ec05cf2e92e0e9a85ff280977b_rle_crop_3973262865_0.png temp/1758877697_42667_1385827604_6981a0ec05cf2e92e0e9a85ff280977b_rle_crop_3973262875_0.png temp/1758877697_42667_1385827604_6981a0ec05cf2e92e0e9a85ff280977b_rle_crop_3973262869_0.png temp/1758877697_42667_1385827604_6981a0ec05cf2e92e0e9a85ff280977b_rle_crop_3973262876_0.png temp/1758877697_42667_1385827571_bde4c433d1056e95accf6113889f54d9_rle_crop_3973262880_0.png temp/1758877697_42667_1385833634_1a0f6be4ce455c7528ed152c6aa16506_rle_crop_3973260005_0.png temp/1758877697_42667_1385830342_b830328de2e84db3f2c61c5897eee07b_rle_crop_3973260027_0.png temp/1758877697_42667_1385829658_805e2de710dc90af5cb441f31ed6b27e_rle_crop_3976616870_0.png temp/1758877697_42667_1385829652_4187f4d15f92a352dab5176f91bd8aaf_rle_crop_3973260041_0.png temp/1758877697_42667_1385829641_6aa441f240453c8550fc21046efebe7d_rle_crop_3973260062_0.png temp/1758877697_42667_1385827780_f3a931ddf32d94053cd9fb345d128c3f_rle_crop_3973260069_0.png temp/1758877697_42667_1385827744_d81372b8b770c6912718ace8b7ced5e0_rle_crop_3973262840_0.png temp/1758877697_42667_1385827706_b90a3384d7d57091d413a90fe0bee31c_rle_crop_3973262855_0.png temp/1758877697_42667_1385830344_1dc22bdc05067dfaf24b0238bd6faa2e_rle_crop_3973260017_0.png Found 86 images belonging to 1 classes. begin to do the prediction : time used to do the prediction : 5.008438587188721 (86,) (86, 10) (86, 1792) shape of features : (86, 1792) shape of new features : (1, 86, 1792) save descriptor for thcl : 3501 time to traite the descriptors : 0.4425952434539795 storage_type for insertDescriptorsMulti : 3 Missing photo l117 : -3973260004 Missing photo l117 : -3973260016 Missing photo l117 : -3973260026 Missing photo l117 : -3973260034 Missing photo l117 : -3973260037 Missing photo l117 : -3973260065 Missing photo l117 : -3973260067 Missing photo l117 : -3973262849 Missing photo l117 : -3973262850 Missing photo l117 : -3973262848 Missing photo l117 : -3973262858 Missing photo l117 : -3973262856 Missing photo l117 : -3973260018 Missing photo l117 : -3973260039 Missing photo l117 : -3973260040 Missing photo l117 : -3973260050 Missing photo l117 : -3973260052 Missing photo l117 : -3973262835 Missing photo l117 : -3973262861 Missing photo l117 : -3973262873 Missing photo l117 : -3973262881 Missing photo l117 : -3973260003 Missing photo l117 : -3973260007 Missing photo l117 : -3973260025 Missing photo l117 : -3973260029 Missing photo l117 : -3973260057 Missing photo l117 : -3973260070 Missing photo l117 : -3973262839 Missing photo l117 : -3973262851 Missing photo l117 : -3973260006 Missing photo l117 : -3973259999 Missing photo l117 : -3973260010 Missing photo l117 : -3973260013 Missing photo l117 : -3973260012 Missing photo l117 : -3973260014 Missing photo l117 : -3973260028 Missing photo l117 : -3973260021 Missing photo l117 : -3973260032 Missing photo l117 : -3973260035 Missing photo l117 : -3973260043 Missing photo l117 : -3973260044 Missing photo l117 : -3973260058 Missing photo l117 : -3973260060 Missing photo l117 : -3973260059 Missing photo l117 : -3973262832 Missing photo l117 : -3976616871 Missing photo l117 : -3973262841 Missing photo l117 : -3973262838 Missing photo l117 : -3973262845 Missing photo l117 : -3973262853 Missing photo l117 : -3973262854 Missing photo l117 : -3973262847 Missing photo l117 : -3973262863 Missing photo l117 : -3973262860 Missing photo l117 : -3973262870 Missing photo l117 : -3973262874 Missing photo l117 : -3973262866 Missing photo l117 : -3973260000 Missing photo l117 : -3973260002 Missing photo l117 : -3973260015 Missing photo l117 : -3973260011 Missing photo l117 : -3973260022 Missing photo l117 : -3973260024 Missing photo l117 : -3973260033 Missing photo l117 : -3973260042 Missing photo l117 : -3973260047 Missing photo l117 : -3973260055 Missing photo l117 : -3973260066 Missing photo l117 : -3973260063 Missing photo l117 : -3973260064 Missing photo l117 : -3973262834 Missing photo l117 : -3973262836 Missing photo l117 : -3973262865 Missing photo l117 : -3973262875 Missing photo l117 : -3973262869 Missing photo l117 : -3973262876 Missing photo l117 : -3973262880 Missing photo l117 : -3973260005 Missing photo l117 : -3973260027 Missing photo l117 : -3976616870 Missing photo l117 : -3973260041 Missing photo l117 : -3973260062 Missing photo l117 : -3973260069 Missing photo l117 : -3973262840 Missing photo l117 : -3973262855 Missing photo l117 : -3973260017 To insert : -3973260004 Missing photo l134 : -3973260004 To insert : -3973260016 Missing photo l134 : -3973260016 To insert : -3973260026 Missing photo l134 : -3973260026 To insert : -3973260034 Missing photo l134 : -3973260034 To insert : -3973260037 Missing photo l134 : -3973260037 To insert : -3973260065 Missing photo l134 : -3973260065 To insert : -3973260067 Missing photo l134 : -3973260067 To insert : -3973262849 Missing photo l134 : -3973262849 To insert : -3973262850 Missing photo l134 : -3973262850 To insert : -3973262848 Missing photo l134 : -3973262848 To insert : -3973262858 Missing photo l134 : -3973262858 To insert : -3973262856 Missing photo l134 : -3973262856 To insert : -3973260018 Missing photo l134 : -3973260018 To insert : -3973260039 Missing photo l134 : -3973260039 To insert : -3973260040 Missing photo l134 : -3973260040 To insert : -3973260050 Missing photo l134 : -3973260050 To insert : -3973260052 Missing photo l134 : -3973260052 To insert : -3973262835 Missing photo l134 : -3973262835 To insert : -3973262861 Missing photo l134 : -3973262861 To insert : -3973262873 Missing photo l134 : -3973262873 To insert : -3973262881 Missing photo l134 : -3973262881 To insert : -3973260003 Missing photo l134 : -3973260003 To insert : -3973260007 Missing photo l134 : -3973260007 To insert : -3973260025 Missing photo l134 : -3973260025 To insert : -3973260029 Missing photo l134 : -3973260029 To insert : -3973260057 Missing photo l134 : -3973260057 To insert : -3973260070 Missing photo l134 : -3973260070 To insert : -3973262839 Missing photo l134 : -3973262839 To insert : -3973262851 Missing photo l134 : -3973262851 To insert : -3973260006 Missing photo l134 : -3973260006 To insert : -3973259999 Missing photo l134 : -3973259999 To insert : -3973260010 Missing photo l134 : -3973260010 To insert : -3973260013 Missing photo l134 : -3973260013 To insert : -3973260012 Missing photo l134 : -3973260012 To insert : -3973260014 Missing photo l134 : -3973260014 To insert : -3973260028 Missing photo l134 : -3973260028 To insert : -3973260021 Missing photo l134 : -3973260021 To insert : -3973260032 Missing photo l134 : -3973260032 To insert : -3973260035 Missing photo l134 : -3973260035 To insert : -3973260043 Missing photo l134 : -3973260043 To insert : -3973260044 Missing photo l134 : -3973260044 To insert : -3973260058 Missing photo l134 : -3973260058 To insert : -3973260060 Missing photo l134 : -3973260060 To insert : -3973260059 Missing photo l134 : -3973260059 To insert : -3973262832 Missing photo l134 : -3973262832 To insert : -3976616871 Missing photo l134 : -3976616871 To insert : -3973262841 Missing photo l134 : -3973262841 To insert : -3973262838 Missing photo l134 : -3973262838 To insert : -3973262845 Missing photo l134 : -3973262845 To insert : -3973262853 Missing photo l134 : -3973262853 To insert : -3973262854 Missing photo l134 : -3973262854 To insert : -3973262847 Missing photo l134 : -3973262847 To insert : -3973262863 Missing photo l134 : -3973262863 To insert : -3973262860 Missing photo l134 : -3973262860 To insert : -3973262870 Missing photo l134 : -3973262870 To insert : -3973262874 Missing photo l134 : -3973262874 To insert : -3973262866 Missing photo l134 : -3973262866 To insert : -3973260000 Missing photo l134 : -3973260000 To insert : -3973260002 Missing photo l134 : -3973260002 To insert : -3973260015 Missing photo l134 : -3973260015 To insert : -3973260011 Missing photo l134 : -3973260011 To insert : -3973260022 Missing photo l134 : -3973260022 To insert : -3973260024 Missing photo l134 : -3973260024 To insert : -3973260033 Missing photo l134 : -3973260033 To insert : -3973260042 Missing photo l134 : -3973260042 To insert : -3973260047 Missing photo l134 : -3973260047 To insert : -3973260055 Missing photo l134 : -3973260055 To insert : -3973260066 Missing photo l134 : -3973260066 To insert : -3973260063 Missing photo l134 : -3973260063 To insert : -3973260064 Missing photo l134 : -3973260064 To insert : -3973262834 Missing photo l134 : -3973262834 To insert : -3973262836 Missing photo l134 : -3973262836 To insert : -3973262865 Missing photo l134 : -3973262865 To insert : -3973262875 Missing photo l134 : -3973262875 To insert : -3973262869 Missing photo l134 : -3973262869 To insert : -3973262876 Missing photo l134 : -3973262876 To insert : -3973262880 Missing photo l134 : -3973262880 To insert : -3973260005 Missing photo l134 : -3973260005 To insert : -3973260027 Missing photo l134 : -3973260027 To insert : -3976616870 Missing photo l134 : -3976616870 To insert : -3973260041 Missing photo l134 : -3973260041 To insert : -3973260062 Missing photo l134 : -3973260062 To insert : -3973260069 Missing photo l134 : -3973260069 To insert : -3973262840 Missing photo l134 : -3973262840 To insert : -3973262855 Missing photo l134 : -3973262855 To insert : -3973260017 Missing photo l134 : -3973260017 time to insert the descriptors : 20.917693614959717 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : tfhub_classification2 we use saveGeneral [1385833634, 1385830344, 1385830342, 1385829658, 1385829652, 1385829650, 1385829644, 1385829643, 1385829641, 1385827780, 1385827744, 1385827706, 1385827651, 1385827604, 1385827571] Looping around the photos to save general results len do output : 86 /-3973260004Didn't retrieve data . /-3973260016Didn't retrieve data . /-3973260026Didn't retrieve data . /-3973260034Didn't retrieve data . /-3973260037Didn't retrieve data . /-3973260065Didn't retrieve data . /-3973260067Didn't retrieve data . /-3973262849Didn't retrieve data . /-3973262850Didn't retrieve data . /-3973262848Didn't retrieve data . /-3973262858Didn't retrieve data . /-3973262856Didn't retrieve data . /-3973260018Didn't retrieve data . /-3973260039Didn't retrieve data . /-3973260040Didn't retrieve data . /-3973260050Didn't retrieve data . /-3973260052Didn't retrieve data . /-3973262835Didn't retrieve data . /-3973262861Didn't retrieve data . /-3973262873Didn't retrieve data . /-3973262881Didn't retrieve data . /-3973260003Didn't retrieve data . /-3973260007Didn't retrieve data . /-3973260025Didn't retrieve data . /-3973260029Didn't retrieve data . /-3973260057Didn't retrieve data . /-3973260070Didn't retrieve data . /-3973262839Didn't retrieve data . /-3973262851Didn't retrieve data . /-3973260006Didn't retrieve data . /-3973259999Didn't retrieve data . /-3973260010Didn't retrieve data . /-3973260013Didn't retrieve data . /-3973260012Didn't retrieve data . /-3973260014Didn't retrieve data . /-3973260028Didn't retrieve data . /-3973260021Didn't retrieve data . /-3973260032Didn't retrieve data . /-3973260035Didn't retrieve data . /-3973260043Didn't retrieve data . /-3973260044Didn't retrieve data . /-3973260058Didn't retrieve data . /-3973260060Didn't retrieve data . /-3973260059Didn't retrieve data . /-3973262832Didn't retrieve data . /-3976616871Didn't retrieve data . /-3973262841Didn't retrieve data . /-3973262838Didn't retrieve data . /-3973262845Didn't retrieve data . /-3973262853Didn't retrieve data . /-3973262854Didn't retrieve data . /-3973262847Didn't retrieve data . /-3973262863Didn't retrieve data . /-3973262860Didn't retrieve data . /-3973262870Didn't retrieve data . /-3973262874Didn't retrieve data . /-3973262866Didn't retrieve data . /-3973260000Didn't retrieve data . /-3973260002Didn't retrieve data . /-3973260015Didn't retrieve data . /-3973260011Didn't retrieve data . /-3973260022Didn't retrieve data . /-3973260024Didn't retrieve data . /-3973260033Didn't retrieve data . /-3973260042Didn't retrieve data . /-3973260047Didn't retrieve data . /-3973260055Didn't retrieve data . /-3973260066Didn't retrieve data . /-3973260063Didn't retrieve data . /-3973260064Didn't retrieve data . /-3973262834Didn't retrieve data . /-3973262836Didn't retrieve data . /-3973262865Didn't retrieve data . /-3973262875Didn't retrieve data . /-3973262869Didn't retrieve data . /-3973262876Didn't retrieve data . /-3973262880Didn't retrieve data . /-3973260005Didn't retrieve data . /-3973260027Didn't retrieve data . /-3976616870Didn't retrieve data . /-3973260041Didn't retrieve data . /-3973260062Didn't retrieve data . /-3973260069Didn't retrieve data . /-3973262840Didn't retrieve data . /-3973262855Didn't retrieve data . /-3973260017Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385833634', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385830344', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385830342', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829658', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829652', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829650', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829644', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829643', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829641', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827780', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827744', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827706', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827651', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827604', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827571', None, None, None, None, None, '3776893') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 101 time used for this insertion : 0.028202295303344727 save_final save missing photos in datou_result : time spend for datou_step_exec : 50.05147123336792 time spend to save output : 0.030173063278198242 total time spend for step 6 : 50.08164429664612 step7:merge_mask_thcl_custom Fri Sep 26 11:13:16 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 128 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1385833634, 1385830344, 1385830342, 1385829658, 1385829652, 1385829650, 1385829644, 1385829643, 1385829641, 1385827780, 1385827744, 1385827706, 1385827651, 1385827604, 1385827571] Looping around the photos to save general results len do output : 15 /1385833634Didn't retrieve data .Didn't retrieve data . /1385830344Didn't retrieve data .Didn't retrieve data . /1385830342Didn't retrieve data .Didn't retrieve data . /1385829658Didn't retrieve data .Didn't retrieve data . /1385829652Didn't retrieve data .Didn't retrieve data . /1385829650Didn't retrieve data .Didn't retrieve data . /1385829644Didn't retrieve data .Didn't retrieve data . /1385829643Didn't retrieve data .Didn't retrieve data . /1385829641Didn't retrieve data .Didn't retrieve data . /1385827780Didn't retrieve data .Didn't retrieve data . /1385827744Didn't retrieve data .Didn't retrieve data . /1385827706Didn't retrieve data .Didn't retrieve data . /1385827651Didn't retrieve data .Didn't retrieve data . /1385827604Didn't retrieve data .Didn't retrieve data . /1385827571Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385833634', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385830344', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385830342', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829658', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829652', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829650', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829644', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829643', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829641', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827780', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827744', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827706', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827651', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827604', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827571', None, None, None, None, None, '3776893') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 45 time used for this insertion : 0.013021707534790039 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.06412172317504883 time spend to save output : 0.01363825798034668 total time spend for step 7 : 0.07775998115539551 step8:rle_unique_nms_with_priority Fri Sep 26 11:13:16 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 9 nb_hashtags : 7 time to prepare the origin masks : 7.6944708824157715 time for calcul the mask position with numpy : 0.34941768646240234 nb_pixel_total : 9253693 time to create 1 rle with new method : 0.7789568901062012 time for calcul the mask position with numpy : 0.0432744026184082 nb_pixel_total : 712457 time to create 1 rle with new method : 0.708737850189209 time for calcul the mask position with numpy : 0.03681135177612305 nb_pixel_total : 40811 time to create 1 rle with old method : 0.044628143310546875 time for calcul the mask position with numpy : 0.036962270736694336 nb_pixel_total : 205491 time to create 1 rle with new method : 0.5808799266815186 time for calcul the mask position with numpy : 0.03820395469665527 nb_pixel_total : 99692 time to create 1 rle with old method : 0.10786318778991699 time for calcul the mask position with numpy : 0.034566402435302734 nb_pixel_total : 19361 time to create 1 rle with old method : 0.021037578582763672 time for calcul the mask position with numpy : 0.03843832015991211 nb_pixel_total : 429856 time to create 1 rle with new method : 0.6386673450469971 time for calcul the mask position with numpy : 0.03564882278442383 nb_pixel_total : 83749 time to create 1 rle with old method : 0.08914041519165039 time for calcul the mask position with numpy : 0.039098262786865234 nb_pixel_total : 547881 time to create 1 rle with new method : 0.587376594543457 time for calcul the mask position with numpy : 0.04288840293884277 nb_pixel_total : 937249 time to create 1 rle with new method : 0.5965383052825928 create new chi : 5.12011456489563 time to delete rle : 0.010050058364868164 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 16065 TO DO : save crop sub photo not yet done ! save time : 0.8726842403411865 nb_obj : 9 nb_hashtags : 5 time to prepare the origin masks : 11.001594066619873 time for calcul the mask position with numpy : 0.42403364181518555 nb_pixel_total : 9643717 time to create 1 rle with new method : 0.537675142288208 time for calcul the mask position with numpy : 0.03737211227416992 nb_pixel_total : 334859 time to create 1 rle with new method : 0.504626989364624 time for calcul the mask position with numpy : 0.0366363525390625 nb_pixel_total : 320106 time to create 1 rle with new method : 0.5597898960113525 time for calcul the mask position with numpy : 0.03653144836425781 nb_pixel_total : 108866 time to create 1 rle with old method : 0.11516547203063965 time for calcul the mask position with numpy : 0.03394365310668945 nb_pixel_total : 52563 time to create 1 rle with old method : 0.05552363395690918 time for calcul the mask position with numpy : 0.03465390205383301 nb_pixel_total : 8923 time to create 1 rle with old method : 0.009594440460205078 time for calcul the mask position with numpy : 0.04264354705810547 nb_pixel_total : 1498718 time to create 1 rle with new method : 0.5449478626251221 time for calcul the mask position with numpy : 0.033701419830322266 nb_pixel_total : 49087 time to create 1 rle with old method : 0.05224752426147461 time for calcul the mask position with numpy : 0.03595280647277832 nb_pixel_total : 292014 time to create 1 rle with new method : 0.5281624794006348 time for calcul the mask position with numpy : 0.03485393524169922 nb_pixel_total : 21387 time to create 1 rle with old method : 0.02307605743408203 create new chi : 3.8840761184692383 time to delete rle : 0.0016071796417236328 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 16765 TO DO : save crop sub photo not yet done ! save time : 0.9324722290039062 nb_obj : 9 nb_hashtags : 7 time to prepare the origin masks : 9.151620626449585 time for calcul the mask position with numpy : 0.4862091541290283 nb_pixel_total : 9253693 time to create 1 rle with new method : 0.5301520824432373 time for calcul the mask position with numpy : 0.04300522804260254 nb_pixel_total : 712457 time to create 1 rle with new method : 0.5252857208251953 time for calcul the mask position with numpy : 0.034316301345825195 nb_pixel_total : 40811 time to create 1 rle with old method : 0.0421292781829834 time for calcul the mask position with numpy : 0.035130977630615234 nb_pixel_total : 205491 time to create 1 rle with new method : 0.5257198810577393 time for calcul the mask position with numpy : 0.03484463691711426 nb_pixel_total : 99692 time to create 1 rle with old method : 0.10420465469360352 time for calcul the mask position with numpy : 0.03348088264465332 nb_pixel_total : 19361 time to create 1 rle with old method : 0.020563602447509766 time for calcul the mask position with numpy : 0.03949737548828125 nb_pixel_total : 429856 time to create 1 rle with new method : 0.584294319152832 time for calcul the mask position with numpy : 0.04062390327453613 nb_pixel_total : 83749 time to create 1 rle with old method : 0.1068875789642334 time for calcul the mask position with numpy : 0.038927316665649414 nb_pixel_total : 547881 time to create 1 rle with new method : 0.5217657089233398 time for calcul the mask position with numpy : 0.044576168060302734 nb_pixel_total : 937249 time to create 1 rle with new method : 0.5823986530303955 create new chi : 4.611543416976929 time to delete rle : 0.0014464855194091797 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 16065 TO DO : save crop sub photo not yet done ! save time : 0.8561315536499023 nb_obj : 4 nb_hashtags : 4 time to prepare the origin masks : 6.189264297485352 time for calcul the mask position with numpy : 0.6087121963500977 nb_pixel_total : 12075172 time to create 1 rle with new method : 0.5824806690216064 time for calcul the mask position with numpy : 0.06738066673278809 nb_pixel_total : 15253 time to create 1 rle with old method : 0.023521900177001953 time for calcul the mask position with numpy : 0.059058189392089844 nb_pixel_total : 49587 time to create 1 rle with old method : 0.0526423454284668 time for calcul the mask position with numpy : 0.06032133102416992 nb_pixel_total : 161476 time to create 1 rle with new method : 0.6898236274719238 time for calcul the mask position with numpy : 0.04796457290649414 nb_pixel_total : 28752 time to create 1 rle with old method : 0.030953407287597656 create new chi : 2.320906400680542 time to delete rle : 0.0008656978607177734 batch 1 Loaded 6 chid ids of type : 3726 Number RLEs to save : 5670 TO DO : save crop sub photo not yet done ! save time : 0.35843610763549805 nb_obj : 5 nb_hashtags : 3 time to prepare the origin masks : 5.288459300994873 time for calcul the mask position with numpy : 0.42322874069213867 nb_pixel_total : 10403871 time to create 1 rle with new method : 0.5448408126831055 time for calcul the mask position with numpy : 0.05773520469665527 nb_pixel_total : 32955 time to create 1 rle with old method : 0.03401684761047363 time for calcul the mask position with numpy : 0.05700230598449707 nb_pixel_total : 105780 time to create 1 rle with old method : 0.10993218421936035 time for calcul the mask position with numpy : 0.05903887748718262 nb_pixel_total : 223567 time to create 1 rle with new method : 0.9759464263916016 time for calcul the mask position with numpy : 0.05749773979187012 nb_pixel_total : 95208 time to create 1 rle with old method : 0.1014401912689209 time for calcul the mask position with numpy : 0.05336761474609375 nb_pixel_total : 1468859 time to create 1 rle with new method : 0.690086841583252 create new chi : 3.2957818508148193 time to delete rle : 0.0016283988952636719 batch 1 Loaded 6 chid ids of type : 3726 Number RLEs to save : 10888 TO DO : save crop sub photo not yet done ! save time : 0.6259076595306396 nb_obj : 3 nb_hashtags : 3 time to prepare the origin masks : 4.622551202774048 time for calcul the mask position with numpy : 0.6507787704467773 nb_pixel_total : 12019306 time to create 1 rle with new method : 0.5119385719299316 time for calcul the mask position with numpy : 0.03810000419616699 nb_pixel_total : 192018 time to create 1 rle with new method : 0.7512226104736328 time for calcul the mask position with numpy : 0.03327131271362305 nb_pixel_total : 24554 time to create 1 rle with old method : 0.025330781936645508 time for calcul the mask position with numpy : 0.035662174224853516 nb_pixel_total : 94362 time to create 1 rle with old method : 0.10401439666748047 create new chi : 2.257769823074341 time to delete rle : 0.001148223876953125 batch 1 Loaded 4 chid ids of type : 3726 Number RLEs to save : 5372 TO DO : save crop sub photo not yet done ! save time : 0.3196895122528076 nb_obj : 5 nb_hashtags : 2 time to prepare the origin masks : 7.4001476764678955 time for calcul the mask position with numpy : 0.4790475368499756 nb_pixel_total : 8998745 time to create 1 rle with new method : 0.6837828159332275 time for calcul the mask position with numpy : 0.05832171440124512 nb_pixel_total : 27206 time to create 1 rle with old method : 0.02879953384399414 time for calcul the mask position with numpy : 0.06547284126281738 nb_pixel_total : 176179 time to create 1 rle with new method : 0.7751858234405518 time for calcul the mask position with numpy : 0.05775332450866699 nb_pixel_total : 24055 time to create 1 rle with old method : 0.026322364807128906 time for calcul the mask position with numpy : 0.0734262466430664 nb_pixel_total : 1832245 time to create 1 rle with new method : 0.6457810401916504 time for calcul the mask position with numpy : 0.04911231994628906 nb_pixel_total : 1271810 time to create 1 rle with new method : 0.7694828510284424 create new chi : 3.8945090770721436 time to delete rle : 0.002549886703491211 batch 1 Loaded 6 chid ids of type : 3726 Number RLEs to save : 16605 TO DO : save crop sub photo not yet done ! save time : 0.9290730953216553 nb_obj : 6 nb_hashtags : 3 time to prepare the origin masks : 6.579683303833008 time for calcul the mask position with numpy : 0.5391805171966553 nb_pixel_total : 10401645 time to create 1 rle with new method : 0.6407392024993896 time for calcul the mask position with numpy : 0.03607964515686035 nb_pixel_total : 178434 time to create 1 rle with new method : 0.7547323703765869 time for calcul the mask position with numpy : 0.035846710205078125 nb_pixel_total : 20158 time to create 1 rle with old method : 0.022640228271484375 time for calcul the mask position with numpy : 0.03563427925109863 nb_pixel_total : 179526 time to create 1 rle with new method : 0.6609060764312744 time for calcul the mask position with numpy : 0.036154985427856445 nb_pixel_total : 2010 time to create 1 rle with old method : 0.0021758079528808594 time for calcul the mask position with numpy : 0.038194894790649414 nb_pixel_total : 525673 time to create 1 rle with new method : 0.6782512664794922 time for calcul the mask position with numpy : 0.04599332809448242 nb_pixel_total : 1022794 time to create 1 rle with new method : 0.6667666435241699 create new chi : 4.400974988937378 time to delete rle : 0.0014188289642333984 batch 1 Loaded 7 chid ids of type : 3726 Number RLEs to save : 10938 TO DO : save crop sub photo not yet done ! save time : 0.6181046962738037 nb_obj : 5 nb_hashtags : 5 time to prepare the origin masks : 6.129034996032715 time for calcul the mask position with numpy : 0.6878125667572021 nb_pixel_total : 11556007 time to create 1 rle with new method : 0.7025942802429199 time for calcul the mask position with numpy : 0.03410220146179199 nb_pixel_total : 118920 time to create 1 rle with old method : 0.12808728218078613 time for calcul the mask position with numpy : 0.03693032264709473 nb_pixel_total : 317584 time to create 1 rle with new method : 0.796137809753418 time for calcul the mask position with numpy : 0.037917375564575195 nb_pixel_total : 98497 time to create 1 rle with old method : 0.11149048805236816 time for calcul the mask position with numpy : 0.06046414375305176 nb_pixel_total : 184375 time to create 1 rle with new method : 0.7543344497680664 time for calcul the mask position with numpy : 0.05648231506347656 nb_pixel_total : 54857 time to create 1 rle with old method : 0.05749082565307617 create new chi : 3.6078248023986816 time to delete rle : 0.0010750293731689453 batch 1 Loaded 6 chid ids of type : 3726 Number RLEs to save : 8750 TO DO : save crop sub photo not yet done ! save time : 0.5006020069122314 nb_obj : 7 nb_hashtags : 4 time to prepare the origin masks : 7.595539331436157 time for calcul the mask position with numpy : 0.5423970222473145 nb_pixel_total : 11437231 time to create 1 rle with new method : 0.5247914791107178 time for calcul the mask position with numpy : 0.033622026443481445 nb_pixel_total : 6615 time to create 1 rle with old method : 0.0073544979095458984 time for calcul the mask position with numpy : 0.03345513343811035 nb_pixel_total : 39099 time to create 1 rle with old method : 0.04042649269104004 time for calcul the mask position with numpy : 0.035835981369018555 nb_pixel_total : 31174 time to create 1 rle with old method : 0.033564090728759766 time for calcul the mask position with numpy : 0.03616189956665039 nb_pixel_total : 157235 time to create 1 rle with new method : 0.6860525608062744 time for calcul the mask position with numpy : 0.03521919250488281 nb_pixel_total : 7987 time to create 1 rle with old method : 0.009757518768310547 time for calcul the mask position with numpy : 0.0397639274597168 nb_pixel_total : 214834 time to create 1 rle with new method : 0.6967799663543701 time for calcul the mask position with numpy : 0.039182424545288086 nb_pixel_total : 436065 time to create 1 rle with new method : 0.5990879535675049 create new chi : 3.5660319328308105 time to delete rle : 0.0010180473327636719 batch 1 Loaded 8 chid ids of type : 3726 Number RLEs to save : 9066 TO DO : save crop sub photo not yet done ! save time : 0.5289058685302734 nb_obj : 11 nb_hashtags : 3 time to prepare the origin masks : 8.878813982009888 time for calcul the mask position with numpy : 0.5614950656890869 nb_pixel_total : 11126337 time to create 1 rle with new method : 0.5966622829437256 time for calcul the mask position with numpy : 0.06290459632873535 nb_pixel_total : 11260 time to create 1 rle with old method : 0.018446922302246094 time for calcul the mask position with numpy : 0.0700373649597168 nb_pixel_total : 695813 time to create 1 rle with new method : 0.5425078868865967 time for calcul the mask position with numpy : 0.044881582260131836 nb_pixel_total : 29624 time to create 1 rle with old method : 0.0354461669921875 time for calcul the mask position with numpy : 0.05161547660827637 nb_pixel_total : 231257 time to create 1 rle with new method : 0.5930054187774658 time for calcul the mask position with numpy : 0.038221120834350586 nb_pixel_total : 30916 time to create 1 rle with old method : 0.03365063667297363 time for calcul the mask position with numpy : 0.037223100662231445 nb_pixel_total : 23344 time to create 1 rle with old method : 0.0250704288482666 time for calcul the mask position with numpy : 0.037095069885253906 nb_pixel_total : 21971 time to create 1 rle with old method : 0.024407386779785156 time for calcul the mask position with numpy : 0.03487873077392578 nb_pixel_total : 4286 time to create 1 rle with old method : 0.01433420181274414 time for calcul the mask position with numpy : 0.03630828857421875 nb_pixel_total : 59792 time to create 1 rle with old method : 0.06475496292114258 time for calcul the mask position with numpy : 0.037156105041503906 nb_pixel_total : 93296 time to create 1 rle with old method : 0.09682011604309082 time for calcul the mask position with numpy : 0.033217430114746094 nb_pixel_total : 2344 time to create 1 rle with old method : 0.002474069595336914 create new chi : 3.239593505859375 time to delete rle : 0.000972747802734375 batch 1 Loaded 14 chid ids of type : 3726 Number RLEs to save : 9893 TO DO : save crop sub photo not yet done ! save time : 0.6334829330444336 nb_obj : 11 nb_hashtags : 5 time to prepare the origin masks : 10.195539712905884 time for calcul the mask position with numpy : 0.5942258834838867 nb_pixel_total : 10300794 time to create 1 rle with new method : 0.6483404636383057 time for calcul the mask position with numpy : 0.035097360610961914 nb_pixel_total : 80353 time to create 1 rle with old method : 0.11914396286010742 time for calcul the mask position with numpy : 0.03499126434326172 nb_pixel_total : 70811 time to create 1 rle with old method : 0.08269500732421875 time for calcul the mask position with numpy : 0.03522014617919922 nb_pixel_total : 2784 time to create 1 rle with old method : 0.0033254623413085938 time for calcul the mask position with numpy : 0.05894756317138672 nb_pixel_total : 82644 time to create 1 rle with old method : 0.09038639068603516 time for calcul the mask position with numpy : 0.05800771713256836 nb_pixel_total : 27461 time to create 1 rle with old method : 0.029509305953979492 time for calcul the mask position with numpy : 0.07841706275939941 nb_pixel_total : 1456344 time to create 1 rle with new method : 0.5676462650299072 time for calcul the mask position with numpy : 0.061614990234375 nb_pixel_total : 116236 time to create 1 rle with old method : 0.12743425369262695 time for calcul the mask position with numpy : 0.05890536308288574 nb_pixel_total : 71011 time to create 1 rle with old method : 0.07381963729858398 time for calcul the mask position with numpy : 0.056023359298706055 nb_pixel_total : 92328 time to create 1 rle with old method : 0.09752607345581055 time for calcul the mask position with numpy : 0.056400299072265625 nb_pixel_total : 19988 time to create 1 rle with old method : 0.021326541900634766 time for calcul the mask position with numpy : 0.05544924736022949 nb_pixel_total : 9486 time to create 1 rle with old method : 0.010303974151611328 create new chi : 3.1642258167266846 time to delete rle : 0.0013041496276855469 batch 1 Loaded 12 chid ids of type : 3726 Number RLEs to save : 14263 TO DO : save crop sub photo not yet done ! save time : 0.7955522537231445 nb_obj : 8 nb_hashtags : 4 time to prepare the origin masks : 7.21442174911499 time for calcul the mask position with numpy : 0.6264457702636719 nb_pixel_total : 11047368 time to create 1 rle with new method : 0.6653802394866943 time for calcul the mask position with numpy : 0.03601980209350586 nb_pixel_total : 77051 time to create 1 rle with old method : 0.08297085762023926 time for calcul the mask position with numpy : 0.03481340408325195 nb_pixel_total : 16179 time to create 1 rle with old method : 0.018174171447753906 time for calcul the mask position with numpy : 0.04040193557739258 nb_pixel_total : 526707 time to create 1 rle with new method : 0.7982118129730225 time for calcul the mask position with numpy : 0.04072165489196777 nb_pixel_total : 2630 time to create 1 rle with old method : 0.0031158924102783203 time for calcul the mask position with numpy : 0.06155562400817871 nb_pixel_total : 510657 time to create 1 rle with new method : 0.796428918838501 time for calcul the mask position with numpy : 0.05995345115661621 nb_pixel_total : 112182 time to create 1 rle with old method : 0.11676430702209473 time for calcul the mask position with numpy : 0.05759239196777344 nb_pixel_total : 26374 time to create 1 rle with old method : 0.029093265533447266 time for calcul the mask position with numpy : 0.05311727523803711 nb_pixel_total : 11092 time to create 1 rle with old method : 0.011565923690795898 create new chi : 3.664594888687134 time to delete rle : 0.001115560531616211 batch 1 Loaded 9 chid ids of type : 3726 Number RLEs to save : 11054 TO DO : save crop sub photo not yet done ! save time : 0.6118731498718262 nb_obj : 14 nb_hashtags : 4 time to prepare the origin masks : 16.352498054504395 time for calcul the mask position with numpy : 0.42865610122680664 nb_pixel_total : 9110234 time to create 1 rle with new method : 0.6428642272949219 time for calcul the mask position with numpy : 0.0417788028717041 nb_pixel_total : 101358 time to create 1 rle with old method : 0.11000490188598633 time for calcul the mask position with numpy : 0.03545498847961426 nb_pixel_total : 73830 time to create 1 rle with old method : 0.08179521560668945 time for calcul the mask position with numpy : 0.04000520706176758 nb_pixel_total : 504684 time to create 1 rle with new method : 0.5768327713012695 time for calcul the mask position with numpy : 0.05764937400817871 nb_pixel_total : 18853 time to create 1 rle with old method : 0.019643545150756836 time for calcul the mask position with numpy : 0.05973505973815918 nb_pixel_total : 167231 time to create 1 rle with new method : 0.736290454864502 time for calcul the mask position with numpy : 0.06348896026611328 nb_pixel_total : 318656 time to create 1 rle with new method : 0.5858609676361084 time for calcul the mask position with numpy : 0.058036088943481445 nb_pixel_total : 230401 time to create 1 rle with new method : 0.5805513858795166 time for calcul the mask position with numpy : 0.05614733695983887 nb_pixel_total : 16763 time to create 1 rle with old method : 0.017642974853515625 time for calcul the mask position with numpy : 0.05780315399169922 nb_pixel_total : 26973 time to create 1 rle with old method : 0.028393268585205078 time for calcul the mask position with numpy : 0.05664682388305664 nb_pixel_total : 49780 time to create 1 rle with old method : 0.05335116386413574 time for calcul the mask position with numpy : 0.06655406951904297 nb_pixel_total : 726569 time to create 1 rle with new method : 0.5932857990264893 time for calcul the mask position with numpy : 0.05962657928466797 nb_pixel_total : 26487 time to create 1 rle with old method : 0.030383825302124023 time for calcul the mask position with numpy : 0.06693291664123535 nb_pixel_total : 955117 time to create 1 rle with new method : 0.5712132453918457 time for calcul the mask position with numpy : 0.05699801445007324 nb_pixel_total : 3304 time to create 1 rle with old method : 0.0036551952362060547 create new chi : 6.126911878585815 time to delete rle : 0.002847909927368164 batch 1 Loaded 16 chid ids of type : 3726 Number RLEs to save : 24417 TO DO : save crop sub photo not yet done ! save time : 1.369051456451416 nb_obj : 5 nb_hashtags : 2 time to prepare the origin masks : 6.58109712600708 time for calcul the mask position with numpy : 0.8116681575775146 nb_pixel_total : 8180873 time to create 1 rle with new method : 0.8235363960266113 time for calcul the mask position with numpy : 0.05778026580810547 nb_pixel_total : 49141 time to create 1 rle with old method : 0.0522913932800293 time for calcul the mask position with numpy : 0.05933380126953125 nb_pixel_total : 100122 time to create 1 rle with old method : 0.11525845527648926 time for calcul the mask position with numpy : 0.04651761054992676 nb_pixel_total : 629916 time to create 1 rle with new method : 0.9078028202056885 time for calcul the mask position with numpy : 0.03842520713806152 nb_pixel_total : 186641 time to create 1 rle with new method : 0.9090511798858643 time for calcul the mask position with numpy : 0.05764007568359375 nb_pixel_total : 3183547 time to create 1 rle with new method : 0.7593812942504883 create new chi : 4.813690900802612 time to delete rle : 0.0021190643310546875 batch 1 Loaded 7 chid ids of type : 3726 Number RLEs to save : 18286 TO DO : save crop sub photo not yet done ! save time : 1.0765767097473145 map_output_result : {1385833634: (0.0, 'Should be the crop_list due to order', 0.0), 1385830344: (0.0, 'Should be the crop_list due to order', 0.0), 1385830342: (0.0, 'Should be the crop_list due to order', 0.0), 1385829658: (0.0, 'Should be the crop_list due to order', 0.0), 1385829652: (0.0, 'Should be the crop_list due to order', 0.0), 1385829650: (0.0, 'Should be the crop_list due to order', 0.0), 1385829644: (0.0, 'Should be the crop_list due to order', 0.0), 1385829643: (0.0, 'Should be the crop_list due to order', 0.0), 1385829641: (0.0, 'Should be the crop_list due to order', 0.0), 1385827780: (0.0, 'Should be the crop_list due to order', 0.0), 1385827744: (0.0, 'Should be the crop_list due to order', 0.0), 1385827706: (0.0, 'Should be the crop_list due to order', 0.0), 1385827651: (0.0, 'Should be the crop_list due to order', 0.0), 1385827604: (0.0, 'Should be the crop_list due to order', 0.0), 1385827571: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1385833634, 1385830344, 1385830342, 1385829658, 1385829652, 1385829650, 1385829644, 1385829643, 1385829641, 1385827780, 1385827744, 1385827706, 1385827651, 1385827604, 1385827571] Looping around the photos to save general results len do output : 15 /1385833634.Didn't retrieve data . /1385830344.Didn't retrieve data . /1385830342.Didn't retrieve data . /1385829658.Didn't retrieve data . /1385829652.Didn't retrieve data . /1385829650.Didn't retrieve data . /1385829644.Didn't retrieve data . /1385829643.Didn't retrieve data . /1385829641.Didn't retrieve data . /1385827780.Didn't retrieve data . /1385827744.Didn't retrieve data . /1385827706.Didn't retrieve data . /1385827651.Didn't retrieve data . /1385827604.Didn't retrieve data . /1385827571.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385833634', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385830344', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385830342', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829658', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829652', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829650', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829644', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829643', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829641', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827780', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827744', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827706', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827651', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827604', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827571', None, None, None, None, None, '3776893') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 45 time used for this insertion : 0.020916461944580078 save_final save missing photos in datou_result : time spend for datou_step_exec : 193.8642349243164 time spend to save output : 0.03634357452392578 total time spend for step 8 : 193.90057849884033 step9:crop_condition Fri Sep 26 11:16:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 15 ! batch 1 Loaded 131 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! About to upload 7 photos upload in portfolio : 4869462 init cache_photo without model_param we have 7 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758878200_42667 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 7 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.9367082118988037 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758878210_42667 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.845977783203125 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758878217_42667 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.2874135971069336 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758878223_42667 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.4063403606414795 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 10 About to insert : list_path_to_insert length 10 new photo from crops ! About to upload 10 photos upload in portfolio : 4869462 init cache_photo without model_param we have 10 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758878230_42667 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 10 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.723156690597534 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758878239_42667 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.315269947052002 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1385833634, 1385830344, 1385830342, 1385829658, 1385829652, 1385829650, 1385829644, 1385829643, 1385829641, 1385827780, 1385827744, 1385827706, 1385827651, 1385827604, 1385827571] Looping around the photos to save general results len do output : 29 /1386328288Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328289Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328290Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328291Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328292Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328293Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328294Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328297Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328298Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328299Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328300Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328301Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328303Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328304Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328305Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328309Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328310Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328311Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328312Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328313Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328314Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328315Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328316Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328317Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328318Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328323Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328324Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328325Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386328326Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385833634', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385830344', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385830342', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829658', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829652', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829650', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829644', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829643', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829641', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827780', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827744', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827706', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827651', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827604', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827571', None, None, None, None, None, '3776893') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 102 time used for this insertion : 0.04337716102600098 save_final save missing photos in datou_result : time spend for datou_step_exec : 51.352394104003906 time spend to save output : 0.0767972469329834 total time spend for step 9 : 51.42919135093689 step10:ventilate_hashtags_in_portfolio Fri Sep 26 11:17:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 27219530 get user id for portfolio 27219530 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27219530 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton_brun','cartonnette','papier','environnement','kraft','Teint_Dans_La_Masse','flou','metal','mal_croppe','Carton_gris','autre_refus','plastique')) AND mptpi.`min_score`=0.5 To do To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27219530 AND mptpi.`type`=4417 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton_brun','cartonnette','papier','environnement','kraft','Teint_Dans_La_Masse','flou','metal','mal_croppe','Carton_gris','autre_refus','plastique')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://marlene.fotonower.com/velours/27219718,27219719,27219720,27219721,27219722,27219723,27219724,27219725,27219726,27219727,27219728,27219729?tags=plastique,papier,mal_croppe,flou,autre_refus,Teint_Dans_La_Masse,environnement,Carton_brun,Carton_gris,kraft,metal,cartonnette Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1385833634, 1385830344, 1385830342, 1385829658, 1385829652, 1385829650, 1385829644, 1385829643, 1385829641, 1385827780, 1385827744, 1385827706, 1385827651, 1385827604, 1385827571] Looping around the photos to save general results len do output : 1 /27219530. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385833634', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385830344', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385830342', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829658', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829652', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829650', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829644', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829643', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829641', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827780', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827744', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827706', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827651', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827604', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827571', None, None, None, None, None, '3776893') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 16 time used for this insertion : 0.017983198165893555 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.5028631687164307 time spend to save output : 0.018372297286987305 total time spend for step 10 : 0.521235466003418 step11:final Fri Sep 26 11:17:22 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1385833634: ('0.11586519449145312',), 1385830344: ('0.11586519449145312',), 1385830342: ('0.11586519449145312',), 1385829658: ('0.11586519449145312',), 1385829652: ('0.11586519449145312',), 1385829650: ('0.11586519449145312',), 1385829644: ('0.11586519449145312',), 1385829643: ('0.11586519449145312',), 1385829641: ('0.11586519449145312',), 1385827780: ('0.11586519449145312',), 1385827744: ('0.11586519449145312',), 1385827706: ('0.11586519449145312',), 1385827651: ('0.11586519449145312',), 1385827604: ('0.11586519449145312',), 1385827571: ('0.11586519449145312',)} new output for save of step final : {1385833634: ('0.11586519449145312',), 1385830344: ('0.11586519449145312',), 1385830342: ('0.11586519449145312',), 1385829658: ('0.11586519449145312',), 1385829652: ('0.11586519449145312',), 1385829650: ('0.11586519449145312',), 1385829644: ('0.11586519449145312',), 1385829643: ('0.11586519449145312',), 1385829641: ('0.11586519449145312',), 1385827780: ('0.11586519449145312',), 1385827744: ('0.11586519449145312',), 1385827706: ('0.11586519449145312',), 1385827651: ('0.11586519449145312',), 1385827604: ('0.11586519449145312',), 1385827571: ('0.11586519449145312',)} [1385833634, 1385830344, 1385830342, 1385829658, 1385829652, 1385829650, 1385829644, 1385829643, 1385829641, 1385827780, 1385827744, 1385827706, 1385827651, 1385827604, 1385827571] Looping around the photos to save general results len do output : 15 /1385833634.Didn't retrieve data . /1385830344.Didn't retrieve data . /1385830342.Didn't retrieve data . /1385829658.Didn't retrieve data . /1385829652.Didn't retrieve data . /1385829650.Didn't retrieve data . /1385829644.Didn't retrieve data . /1385829643.Didn't retrieve data . /1385829641.Didn't retrieve data . /1385827780.Didn't retrieve data . /1385827744.Didn't retrieve data . /1385827706.Didn't retrieve data . /1385827651.Didn't retrieve data . /1385827604.Didn't retrieve data . /1385827571.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385833634', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385830344', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385830342', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829658', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829652', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829650', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829644', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829643', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829641', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827780', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827744', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827706', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827651', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827604', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827571', None, None, None, None, None, '3776893') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 45 time used for this insertion : 0.014146804809570312 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.14861726760864258 time spend to save output : 0.014858245849609375 total time spend for step 11 : 0.16347551345825195 step12:velours_tree Fri Sep 26 11:17:22 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.26998090744018555 time spend to save output : 3.457069396972656e-05 total time spend for step 12 : 0.2700154781341553 step13:send_mail_cod Fri Sep 26 11:17:22 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin in order to get the selector url, please entre the license of selector results_Auto_P27219530_26-09-2025_11_17_22.pdf 27219705 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272197051758878242 27219707 imagette272197071758878244 27219708 imagette272197081758878244 27219709 change filename to text .change filename to text .change filename to text .change filename to text .imagette272197091758878244 27219714 imagette272197141758878244 27219715 imagette272197151758878244 27219716 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272197161758878244 27219710 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272197101758878246 27219712 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272197121758878248 27219713 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272197131758878251 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=27219530 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://marlene.fotonower.com/velours/27219718,27219719,27219720,27219721,27219722,27219723,27219724,27219725,27219726,27219727,27219728,27219729?tags=plastique,papier,mal_croppe,flou,autre_refus,Teint_Dans_La_Masse,environnement,Carton_brun,Carton_gris,kraft,metal,cartonnette your option no_mail is active, we will not send the real mail to your client args[1385833634] : ((1385833634, -0.05263166378289225, 2107752395), (1385833634, -0.11931854051268295, 492688767), '0.11586519449145312') We are sending mail with results at report@fotonower.com args[1385830344] : ((1385830344, -0.2508248374858708, 496442774), (1385830344, -0.16936313796585276, 492688767), '0.11586519449145312') We are sending mail with results at report@fotonower.com args[1385830342] : ((1385830342, -0.05263166378289225, 2107752395), (1385830342, -0.11931854051268295, 492688767), '0.11586519449145312') We are sending mail with results at report@fotonower.com args[1385829658] : ((1385829658, 0.026141028634796004, 2107752395), (1385829658, 0.1650984531302115, 492688767), '0.11586519449145312') We are sending mail with results at report@fotonower.com args[1385829652] : ((1385829652, -0.1418021520749051, 496442774), (1385829652, -0.3955859427101483, 492688767), '0.11586519449145312') We are sending mail with results at report@fotonower.com args[1385829650] : ((1385829650, 0.12395000553779005, 2107752395), (1385829650, -0.11164768424730849, 492688767), '0.11586519449145312') We are sending mail with results at report@fotonower.com args[1385829644] : ((1385829644, 0.0467587355019474, 2107752395), (1385829644, -0.022966920474244326, 492688767), '0.11586519449145312') We are sending mail with results at report@fotonower.com args[1385829643] : ((1385829643, 0.220628821986616, 2107752395), (1385829643, 0.0796464882647771, 492688767), '0.11586519449145312') We are sending mail with results at report@fotonower.com args[1385829641] : ((1385829641, 0.14711290796332052, 2107752395), (1385829641, 0.10237068577841801, 492688767), '0.11586519449145312') We are sending mail with results at report@fotonower.com args[1385827780] : ((1385827780, 0.05796401497520871, 2107752395), (1385827780, 0.5909988218980617, 492688767), '0.11586519449145312') We are sending mail with results at report@fotonower.com args[1385827744] : ((1385827744, -0.06051899045539021, 2107752395), (1385827744, 0.8341130890569599, 492688767), '0.11586519449145312') We are sending mail with results at report@fotonower.com args[1385827706] : ((1385827706, 0.04070033708440614, 2107752395), (1385827706, 0.7055980093397689, 492688767), '0.11586519449145312') We are sending mail with results at report@fotonower.com args[1385827651] : ((1385827651, 0.37999373726859326, 2107752395), (1385827651, 0.18466337863131607, 492688767), '0.11586519449145312') We are sending mail with results at report@fotonower.com args[1385827604] : ((1385827604, 0.0658006091046822, 2107752395), (1385827604, 0.3101181541870158, 492688767), '0.11586519449145312') We are sending mail with results at report@fotonower.com args[1385827571] : ((1385827571, -0.5607209864397579, 501862349), (1385827571, -0.08391098891371514, 492688767), '0.11586519449145312') We are sending mail with results at report@fotonower.com refus_total : 0.11586519449145312 2022-04-13 10:29:59 0 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27219530_26-09-2025_11_17_22.pdf results_Auto_P27219530_26-09-2025_11_17_22.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27219530_26-09-2025_11_17_22.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4433','27219530','results_Auto_P27219530_26-09-2025_11_17_22.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27219530_26-09-2025_11_17_22.pdf','pdf','','1.43','0.11586519449145312') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1385833634, 1385830344, 1385830342, 1385829658, 1385829652, 1385829650, 1385829644, 1385829643, 1385829641, 1385827780, 1385827744, 1385827706, 1385827651, 1385827604, 1385827571] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385833634', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385830344', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385830342', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829658', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829652', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829650', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829644', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829643', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829641', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827780', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827744', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827706', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827651', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827604', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827571', None, None, None, None, None, '3776893') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 15 time used for this insertion : 0.018012285232543945 save_final save missing photos in datou_result : time spend for datou_step_exec : 14.130830526351929 time spend to save output : 0.018351078033447266 total time spend for step 13 : 14.149181604385376 step14:split_time_score Fri Sep 26 11:17:37 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('14', 89),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 18092025 27219530 Nombre de photos uploadées : 89 / 23040 (0%) 18092025 27219530 Nombre de photos taguées (types de déchets): 0 / 89 (0%) 18092025 27219530 Nombre de photos taguées (volume) : 0 / 89 (0%) elapsed_time : load_data_split_time_score 2.384185791015625e-06 elapsed_time : order_list_meta_photo_and_scores 6.4373016357421875e-06 ????????????????????????????????????????????????????????????????????????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.004147052764892578 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.21009087562561035 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.15401314081438286 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26998004_18-09-2025_12_55_19.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26998004 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26998004 AND mptpi.`type`=3726 To do Qualite : 0.15876110650577957 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26999382_18-09-2025_16_28_13.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26999382 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26999382 AND mptpi.`type`=3726 To do Qualite : 0.10113798898534726 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27000904_18-09-2025_16_06_10.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27000904 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27000904 AND mptpi.`type`=3726 To do Qualite : 0.18364316406527295 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27120288_23-09-2025_22_18_47.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27120288 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27120288 AND mptpi.`type`=3726 To do Qualite : 0.05787806829603807 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27124812_23-09-2025_13_23_41.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27124812 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27124812 AND mptpi.`type`=3726 To do Qualite : 0.09959378803985038 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27172550_24-09-2025_13_54_49.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27172550 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27172550 AND mptpi.`type`=3726 To do Qualite : 0.1160095068060618 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27219530_26-09-2025_11_17_22.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27219530 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27219530 AND mptpi.`type`=3726 To do Qualite : 0.10822141120346863 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27268665_26-09-2025_10_45_57.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27268665 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27268665 AND mptpi.`type`=3726 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'18092025': {'nb_upload': 89, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1385833634, 1385830344, 1385830342, 1385829658, 1385829652, 1385829650, 1385829644, 1385829643, 1385829641, 1385827780, 1385827744, 1385827706, 1385827651, 1385827604, 1385827571] Looping around the photos to save general results len do output : 1 /27219530Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385833634', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385830344', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385830342', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829658', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829652', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829650', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829644', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829643', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385829641', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827780', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827744', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827706', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827651', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827604', None, None, None, None, None, '3776893') ('4433', None, None, None, None, None, None, None, '3776893') ('4433', None, '1385827571', None, None, None, None, None, '3776893') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 16 time used for this insertion : 0.014566898345947266 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.9723656177520752 time spend to save output : 0.01485300064086914 total time spend for step 14 : 0.9872186183929443 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 270.54user 236.59system 9:29.41elapsed 89%CPU (0avgtext+0avgdata 5683712maxresident)k 7025208inputs+433000outputs (129747major+31404645minor)pagefaults 0swaps