python /home/admin/mtr/script_for_cron.py -j default -m 20 -a 'python3 ~/workarea/git/Velours/python/prod/datou.py -j batch_current -a 4433' -s traitement_sts -M 0 -S 0 -U 100,80,95 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/home/admin/workarea/git/apy', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 83458 load datou : 4433 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4433, datou_cur_ids : ['3772638'] with mtr_portfolio_ids : ['27193435'] and first list_photo_ids : [] new path : /proc/83458/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, brightness, blur_detection, crop_condition, thcl, tfhub_classification2, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 15 list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 15 ; length of list_pids : 15 ; length of list_args : 15 time to download the photos : 4.987778663635254 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 14 step1:mask_detect Fri Sep 26 11:40:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10582 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-09-26 11:40:34.724640: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-26 11:40:34.752592: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-26 11:40:34.754312: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f4234000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-26 11:40:34.754333: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-26 11:40:34.757125: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-26 11:40:34.947364: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x3c60c3f0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-26 11:40:34.947410: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-26 11:40:34.948197: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:40:34.948545: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:40:34.950697: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:40:34.952821: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:40:34.953173: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:40:34.955568: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:40:34.956640: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:40:34.961244: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:40:34.962685: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:40:34.962772: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:40:34.963540: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-26 11:40:34.963556: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-26 11:40:34.963565: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-26 11:40:34.965551: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9801 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-09-26 11:40:35.250402: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:40:35.250489: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:40:35.250510: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:40:35.250529: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:40:35.250547: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:40:35.250564: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:40:35.250582: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:40:35.250600: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:40:35.252172: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:40:35.253470: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:40:35.253506: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:40:35.253525: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:40:35.253542: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:40:35.253559: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:40:35.253576: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:40:35.253593: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:40:35.253610: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:40:35.255170: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:40:35.255212: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-26 11:40:35.255223: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-26 11:40:35.255233: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-26 11:40:35.256894: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9801 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-09-26 11:40:51.209027: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:40:51.466583: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 15 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 1.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 13 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 13 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 6.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 21 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 11 Detection mask done ! Trying to reset tf kernel 83759 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1841 tf kernel not reseted sub process len(results) : 15 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 15 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 6800 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.0009438991546630859 nb_pixel_total : 36947 time to create 1 rle with old method : 0.04460406303405762 length of segment : 236 time for calcul the mask position with numpy : 0.013630390167236328 nb_pixel_total : 423883 time to create 1 rle with new method : 0.0390772819519043 length of segment : 744 time for calcul the mask position with numpy : 0.0007994174957275391 nb_pixel_total : 48681 time to create 1 rle with old method : 0.05277657508850098 length of segment : 341 time for calcul the mask position with numpy : 0.0001285076141357422 nb_pixel_total : 4447 time to create 1 rle with old method : 0.004845619201660156 length of segment : 133 time for calcul the mask position with numpy : 0.0009524822235107422 nb_pixel_total : 64024 time to create 1 rle with old method : 0.06841421127319336 length of segment : 225 time for calcul the mask position with numpy : 0.0026879310607910156 nb_pixel_total : 176066 time to create 1 rle with new method : 0.010461807250976562 length of segment : 761 time for calcul the mask position with numpy : 0.0006809234619140625 nb_pixel_total : 33046 time to create 1 rle with old method : 0.035453081130981445 length of segment : 161 length of segment : 0 time for calcul the mask position with numpy : 0.06914281845092773 nb_pixel_total : 1473893 time to create 1 rle with new method : 0.830636739730835 length of segment : 2623 time for calcul the mask position with numpy : 0.02324199676513672 nb_pixel_total : 1310221 time to create 1 rle with new method : 0.1865839958190918 length of segment : 2297 time for calcul the mask position with numpy : 0.00022125244140625 nb_pixel_total : 4114 time to create 1 rle with old method : 0.0049021244049072266 length of segment : 95 time for calcul the mask position with numpy : 0.0017879009246826172 nb_pixel_total : 58471 time to create 1 rle with old method : 0.0666813850402832 length of segment : 293 time for calcul the mask position with numpy : 0.0030558109283447266 nb_pixel_total : 90045 time to create 1 rle with old method : 0.10165786743164062 length of segment : 495 time for calcul the mask position with numpy : 0.00121307373046875 nb_pixel_total : 52281 time to create 1 rle with old method : 0.0667123794555664 length of segment : 272 time for calcul the mask position with numpy : 0.0009517669677734375 nb_pixel_total : 47299 time to create 1 rle with old method : 0.05386829376220703 length of segment : 259 time for calcul the mask position with numpy : 0.0003886222839355469 nb_pixel_total : 10606 time to create 1 rle with old method : 0.012416839599609375 length of segment : 144 time for calcul the mask position with numpy : 0.0005044937133789062 nb_pixel_total : 16233 time to create 1 rle with old method : 0.019410371780395508 length of segment : 120 time for calcul the mask position with numpy : 0.03806781768798828 nb_pixel_total : 1506412 time to create 1 rle with new method : 0.42336130142211914 length of segment : 2456 time for calcul the mask position with numpy : 0.003031015396118164 nb_pixel_total : 125701 time to create 1 rle with old method : 0.14005661010742188 length of segment : 453 time for calcul the mask position with numpy : 0.005729198455810547 nb_pixel_total : 293276 time to create 1 rle with new method : 0.0109100341796875 length of segment : 674 time for calcul the mask position with numpy : 0.0024025440216064453 nb_pixel_total : 82677 time to create 1 rle with old method : 0.09461712837219238 length of segment : 467 time for calcul the mask position with numpy : 0.00029540061950683594 nb_pixel_total : 8634 time to create 1 rle with old method : 0.010371923446655273 length of segment : 70 time for calcul the mask position with numpy : 0.0006365776062011719 nb_pixel_total : 28040 time to create 1 rle with old method : 0.033519744873046875 length of segment : 123 time for calcul the mask position with numpy : 0.00041937828063964844 nb_pixel_total : 14615 time to create 1 rle with old method : 0.017564058303833008 length of segment : 159 time for calcul the mask position with numpy : 0.0018608570098876953 nb_pixel_total : 100422 time to create 1 rle with old method : 0.11590075492858887 length of segment : 310 time for calcul the mask position with numpy : 0.0009427070617675781 nb_pixel_total : 31580 time to create 1 rle with old method : 0.036278486251831055 length of segment : 307 time for calcul the mask position with numpy : 0.0027937889099121094 nb_pixel_total : 171656 time to create 1 rle with new method : 0.005048513412475586 length of segment : 396 time for calcul the mask position with numpy : 0.002413511276245117 nb_pixel_total : 113817 time to create 1 rle with old method : 0.12926983833312988 length of segment : 494 time for calcul the mask position with numpy : 0.0013997554779052734 nb_pixel_total : 70618 time to create 1 rle with old method : 0.08037853240966797 length of segment : 263 time for calcul the mask position with numpy : 0.001073598861694336 nb_pixel_total : 29549 time to create 1 rle with old method : 0.0341486930847168 length of segment : 478 time for calcul the mask position with numpy : 0.0010907649993896484 nb_pixel_total : 41451 time to create 1 rle with old method : 0.047385454177856445 length of segment : 274 time for calcul the mask position with numpy : 0.003496408462524414 nb_pixel_total : 239460 time to create 1 rle with new method : 0.0055620670318603516 length of segment : 444 time for calcul the mask position with numpy : 0.0050504207611083984 nb_pixel_total : 303925 time to create 1 rle with new method : 0.007947206497192383 length of segment : 717 time for calcul the mask position with numpy : 0.0005791187286376953 nb_pixel_total : 15766 time to create 1 rle with old method : 0.018268585205078125 length of segment : 142 time for calcul the mask position with numpy : 0.052129507064819336 nb_pixel_total : 1579391 time to create 1 rle with new method : 0.2963433265686035 length of segment : 2581 time for calcul the mask position with numpy : 0.00153350830078125 nb_pixel_total : 36701 time to create 1 rle with old method : 0.04224801063537598 length of segment : 328 time for calcul the mask position with numpy : 0.0006356239318847656 nb_pixel_total : 24491 time to create 1 rle with old method : 0.028615236282348633 length of segment : 124 time for calcul the mask position with numpy : 0.011710882186889648 nb_pixel_total : 636598 time to create 1 rle with new method : 0.03370094299316406 length of segment : 878 time for calcul the mask position with numpy : 0.0005853176116943359 nb_pixel_total : 16407 time to create 1 rle with old method : 0.019714832305908203 length of segment : 109 time for calcul the mask position with numpy : 0.0030634403228759766 nb_pixel_total : 114544 time to create 1 rle with old method : 0.12913990020751953 length of segment : 588 time for calcul the mask position with numpy : 0.0017728805541992188 nb_pixel_total : 84899 time to create 1 rle with old method : 0.09521484375 length of segment : 375 time for calcul the mask position with numpy : 0.0008144378662109375 nb_pixel_total : 31779 time to create 1 rle with old method : 0.03740572929382324 length of segment : 103 time for calcul the mask position with numpy : 0.006650447845458984 nb_pixel_total : 270505 time to create 1 rle with new method : 0.012006282806396484 length of segment : 814 time for calcul the mask position with numpy : 0.002893209457397461 nb_pixel_total : 133663 time to create 1 rle with old method : 0.14862871170043945 length of segment : 362 time for calcul the mask position with numpy : 0.0013394355773925781 nb_pixel_total : 46280 time to create 1 rle with old method : 0.05307912826538086 length of segment : 302 time for calcul the mask position with numpy : 0.0013759136199951172 nb_pixel_total : 86216 time to create 1 rle with old method : 0.09806060791015625 length of segment : 491 time for calcul the mask position with numpy : 0.0008931159973144531 nb_pixel_total : 35962 time to create 1 rle with old method : 0.04117870330810547 length of segment : 244 time for calcul the mask position with numpy : 0.0029053688049316406 nb_pixel_total : 128670 time to create 1 rle with old method : 0.14713048934936523 length of segment : 584 time for calcul the mask position with numpy : 0.022701501846313477 nb_pixel_total : 1139705 time to create 1 rle with new method : 0.2527120113372803 length of segment : 2225 time for calcul the mask position with numpy : 0.00038886070251464844 nb_pixel_total : 8516 time to create 1 rle with old method : 0.009842395782470703 length of segment : 116 time for calcul the mask position with numpy : 0.008682727813720703 nb_pixel_total : 413253 time to create 1 rle with new method : 0.031072139739990234 length of segment : 595 time for calcul the mask position with numpy : 0.0050868988037109375 nb_pixel_total : 392808 time to create 1 rle with new method : 0.009965181350708008 length of segment : 545 time for calcul the mask position with numpy : 0.0006005764007568359 nb_pixel_total : 15766 time to create 1 rle with old method : 0.018741607666015625 length of segment : 142 time for calcul the mask position with numpy : 0.05144095420837402 nb_pixel_total : 1579436 time to create 1 rle with new method : 0.43072962760925293 length of segment : 2581 time for calcul the mask position with numpy : 0.001260519027709961 nb_pixel_total : 36702 time to create 1 rle with old method : 0.0420377254486084 length of segment : 328 time for calcul the mask position with numpy : 0.0006506443023681641 nb_pixel_total : 24490 time to create 1 rle with old method : 0.029137372970581055 length of segment : 124 time for calcul the mask position with numpy : 0.008393287658691406 nb_pixel_total : 636611 time to create 1 rle with new method : 0.022215604782104492 length of segment : 878 time for calcul the mask position with numpy : 0.0002932548522949219 nb_pixel_total : 16405 time to create 1 rle with old method : 0.0195615291595459 length of segment : 108 time for calcul the mask position with numpy : 0.0029289722442626953 nb_pixel_total : 114563 time to create 1 rle with old method : 0.1307849884033203 length of segment : 589 time for calcul the mask position with numpy : 0.0017080307006835938 nb_pixel_total : 84890 time to create 1 rle with old method : 0.09700179100036621 length of segment : 375 time for calcul the mask position with numpy : 0.000789642333984375 nb_pixel_total : 31781 time to create 1 rle with old method : 0.036992788314819336 length of segment : 103 time for calcul the mask position with numpy : 0.004786014556884766 nb_pixel_total : 270495 time to create 1 rle with new method : 0.011113882064819336 length of segment : 814 time for calcul the mask position with numpy : 0.0023488998413085938 nb_pixel_total : 133668 time to create 1 rle with old method : 0.15099811553955078 length of segment : 362 time for calcul the mask position with numpy : 0.0016405582427978516 nb_pixel_total : 86214 time to create 1 rle with old method : 0.10555720329284668 length of segment : 491 time for calcul the mask position with numpy : 0.0013365745544433594 nb_pixel_total : 46273 time to create 1 rle with old method : 0.05342507362365723 length of segment : 301 time for calcul the mask position with numpy : 0.0009102821350097656 nb_pixel_total : 35962 time to create 1 rle with old method : 0.04263567924499512 length of segment : 244 time for calcul the mask position with numpy : 0.002853870391845703 nb_pixel_total : 128647 time to create 1 rle with old method : 0.145430326461792 length of segment : 585 time for calcul the mask position with numpy : 0.023733854293823242 nb_pixel_total : 1139693 time to create 1 rle with new method : 0.20669794082641602 length of segment : 2225 time for calcul the mask position with numpy : 0.0003707408905029297 nb_pixel_total : 8516 time to create 1 rle with old method : 0.009875059127807617 length of segment : 116 time for calcul the mask position with numpy : 0.008485794067382812 nb_pixel_total : 413222 time to create 1 rle with new method : 0.03069901466369629 length of segment : 595 time for calcul the mask position with numpy : 0.0078046321868896484 nb_pixel_total : 392810 time to create 1 rle with new method : 0.010770797729492188 length of segment : 545 time for calcul the mask position with numpy : 0.0012216567993164062 nb_pixel_total : 44092 time to create 1 rle with old method : 0.050699472427368164 length of segment : 182 time for calcul the mask position with numpy : 0.0006575584411621094 nb_pixel_total : 12734 time to create 1 rle with old method : 0.014725446701049805 length of segment : 181 time for calcul the mask position with numpy : 0.00040912628173828125 nb_pixel_total : 8254 time to create 1 rle with old method : 0.010126352310180664 length of segment : 94 time for calcul the mask position with numpy : 0.003402233123779297 nb_pixel_total : 124269 time to create 1 rle with old method : 0.1410846710205078 length of segment : 544 time for calcul the mask position with numpy : 0.0015172958374023438 nb_pixel_total : 31461 time to create 1 rle with old method : 0.03628206253051758 length of segment : 245 time for calcul the mask position with numpy : 0.0009353160858154297 nb_pixel_total : 40324 time to create 1 rle with old method : 0.04640793800354004 length of segment : 193 time for calcul the mask position with numpy : 0.004219770431518555 nb_pixel_total : 180841 time to create 1 rle with new method : 0.006912946701049805 length of segment : 533 time for calcul the mask position with numpy : 0.003354787826538086 nb_pixel_total : 109045 time to create 1 rle with old method : 0.1249995231628418 length of segment : 433 time for calcul the mask position with numpy : 0.005894899368286133 nb_pixel_total : 216119 time to create 1 rle with new method : 0.010305166244506836 length of segment : 531 time for calcul the mask position with numpy : 0.0032045841217041016 nb_pixel_total : 81623 time to create 1 rle with old method : 0.0918278694152832 length of segment : 520 time for calcul the mask position with numpy : 0.0010967254638671875 nb_pixel_total : 40643 time to create 1 rle with old method : 0.045546770095825195 length of segment : 246 time for calcul the mask position with numpy : 0.002677440643310547 nb_pixel_total : 81928 time to create 1 rle with old method : 0.09376668930053711 length of segment : 310 time for calcul the mask position with numpy : 0.0009114742279052734 nb_pixel_total : 17276 time to create 1 rle with old method : 0.023459434509277344 length of segment : 158 time for calcul the mask position with numpy : 0.0303497314453125 nb_pixel_total : 676395 time to create 1 rle with new method : 0.1953890323638916 length of segment : 2119 time for calcul the mask position with numpy : 0.00464630126953125 nb_pixel_total : 184007 time to create 1 rle with new method : 0.007954120635986328 length of segment : 470 time for calcul the mask position with numpy : 0.0014452934265136719 nb_pixel_total : 65923 time to create 1 rle with old method : 0.07527899742126465 length of segment : 259 time for calcul the mask position with numpy : 0.0027730464935302734 nb_pixel_total : 159917 time to create 1 rle with new method : 0.006791830062866211 length of segment : 426 time for calcul the mask position with numpy : 0.0012042522430419922 nb_pixel_total : 24331 time to create 1 rle with old method : 0.028045654296875 length of segment : 501 time for calcul the mask position with numpy : 0.0010886192321777344 nb_pixel_total : 26171 time to create 1 rle with old method : 0.03241395950317383 length of segment : 204 time for calcul the mask position with numpy : 0.006491422653198242 nb_pixel_total : 154511 time to create 1 rle with new method : 0.009490013122558594 length of segment : 505 time for calcul the mask position with numpy : 0.0007424354553222656 nb_pixel_total : 11527 time to create 1 rle with old method : 0.013245105743408203 length of segment : 214 time for calcul the mask position with numpy : 0.006924629211425781 nb_pixel_total : 181556 time to create 1 rle with new method : 0.010803699493408203 length of segment : 531 time for calcul the mask position with numpy : 0.00664830207824707 nb_pixel_total : 182073 time to create 1 rle with new method : 0.010442256927490234 length of segment : 678 time for calcul the mask position with numpy : 0.0012993812561035156 nb_pixel_total : 36905 time to create 1 rle with old method : 0.04590916633605957 length of segment : 135 time for calcul the mask position with numpy : 0.0007038116455078125 nb_pixel_total : 16393 time to create 1 rle with old method : 0.023435115814208984 length of segment : 172 time for calcul the mask position with numpy : 0.0010502338409423828 nb_pixel_total : 37827 time to create 1 rle with old method : 0.05210757255554199 length of segment : 133 time for calcul the mask position with numpy : 0.0011925697326660156 nb_pixel_total : 33636 time to create 1 rle with old method : 0.0406794548034668 length of segment : 255 time for calcul the mask position with numpy : 0.009390830993652344 nb_pixel_total : 326527 time to create 1 rle with new method : 0.013019084930419922 length of segment : 523 time for calcul the mask position with numpy : 0.003016233444213867 nb_pixel_total : 82122 time to create 1 rle with old method : 0.09514260292053223 length of segment : 572 time for calcul the mask position with numpy : 0.007300376892089844 nb_pixel_total : 324195 time to create 1 rle with new method : 0.015469551086425781 length of segment : 519 time for calcul the mask position with numpy : 0.050997257232666016 nb_pixel_total : 1680556 time to create 1 rle with new method : 0.14380621910095215 length of segment : 2499 time for calcul the mask position with numpy : 0.002829313278198242 nb_pixel_total : 93386 time to create 1 rle with old method : 0.13030552864074707 length of segment : 300 time for calcul the mask position with numpy : 0.0008280277252197266 nb_pixel_total : 24239 time to create 1 rle with old method : 0.028510093688964844 length of segment : 187 time for calcul the mask position with numpy : 0.0010612010955810547 nb_pixel_total : 33265 time to create 1 rle with old method : 0.03866219520568848 length of segment : 258 time for calcul the mask position with numpy : 0.0013747215270996094 nb_pixel_total : 59954 time to create 1 rle with old method : 0.0693962574005127 length of segment : 264 time for calcul the mask position with numpy : 0.0024785995483398438 nb_pixel_total : 104469 time to create 1 rle with old method : 0.12481808662414551 length of segment : 432 time for calcul the mask position with numpy : 0.003559112548828125 nb_pixel_total : 158945 time to create 1 rle with new method : 0.005424022674560547 length of segment : 413 time for calcul the mask position with numpy : 0.002201080322265625 nb_pixel_total : 111588 time to create 1 rle with old method : 0.14297747611999512 length of segment : 425 time for calcul the mask position with numpy : 0.08820033073425293 nb_pixel_total : 133032 time to create 1 rle with old method : 0.15115928649902344 length of segment : 446 time for calcul the mask position with numpy : 0.005435466766357422 nb_pixel_total : 46621 time to create 1 rle with old method : 0.055124521255493164 length of segment : 294 time for calcul the mask position with numpy : 0.0037925243377685547 nb_pixel_total : 24807 time to create 1 rle with old method : 0.0318751335144043 length of segment : 192 time for calcul the mask position with numpy : 0.010537147521972656 nb_pixel_total : 121672 time to create 1 rle with old method : 0.14993810653686523 length of segment : 382 time for calcul the mask position with numpy : 0.002848386764526367 nb_pixel_total : 121269 time to create 1 rle with old method : 0.16223430633544922 length of segment : 314 time for calcul the mask position with numpy : 0.0027954578399658203 nb_pixel_total : 60398 time to create 1 rle with old method : 0.06923294067382812 length of segment : 404 time for calcul the mask position with numpy : 0.0017414093017578125 nb_pixel_total : 59054 time to create 1 rle with old method : 0.06711053848266602 length of segment : 348 time for calcul the mask position with numpy : 0.06718945503234863 nb_pixel_total : 775784 time to create 1 rle with new method : 0.27509355545043945 length of segment : 2020 time for calcul the mask position with numpy : 0.0700826644897461 nb_pixel_total : 233826 time to create 1 rle with new method : 0.00954580307006836 length of segment : 917 time for calcul the mask position with numpy : 0.007025718688964844 nb_pixel_total : 314665 time to create 1 rle with new method : 0.01484537124633789 length of segment : 1147 time for calcul the mask position with numpy : 0.007230281829833984 nb_pixel_total : 126484 time to create 1 rle with old method : 0.15732765197753906 length of segment : 597 time for calcul the mask position with numpy : 0.052143096923828125 nb_pixel_total : 33013 time to create 1 rle with old method : 0.049221038818359375 length of segment : 172 time for calcul the mask position with numpy : 0.0007293224334716797 nb_pixel_total : 33197 time to create 1 rle with old method : 0.03824210166931152 length of segment : 168 time for calcul the mask position with numpy : 0.002440214157104492 nb_pixel_total : 28447 time to create 1 rle with old method : 0.04235243797302246 length of segment : 319 time for calcul the mask position with numpy : 0.002346515655517578 nb_pixel_total : 5374 time to create 1 rle with old method : 0.006544828414916992 length of segment : 91 time for calcul the mask position with numpy : 0.03573489189147949 nb_pixel_total : 40823 time to create 1 rle with old method : 0.05350899696350098 length of segment : 413 time for calcul the mask position with numpy : 0.028639793395996094 nb_pixel_total : 43564 time to create 1 rle with old method : 0.06244063377380371 length of segment : 212 time for calcul the mask position with numpy : 0.008330345153808594 nb_pixel_total : 25656 time to create 1 rle with old method : 0.03451347351074219 length of segment : 192 time for calcul the mask position with numpy : 0.12744975090026855 nb_pixel_total : 1030167 time to create 1 rle with new method : 0.33248019218444824 length of segment : 2271 time for calcul the mask position with numpy : 0.0044171810150146484 nb_pixel_total : 12360 time to create 1 rle with old method : 0.016878128051757812 length of segment : 103 time spent for convertir_results : 25.351016998291016 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 129 chid ids of type : 3663 Number RLEs to save : 68462 save missing photos in datou_result : time spend for datou_step_exec : 97.08274745941162 time spend to save output : 3.761796712875366 total time spend for step 1 : 100.84454417228699 step2:brightness Fri Sep 26 11:42:12 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1758879626_83458_1385917724_285359095633e41fa610a45583900295.jpg treat image : temp/1758879626_83458_1385917717_9cc153cbddc93da0a48d94c3edf0fea8.jpg treat image : temp/1758879626_83458_1385917713_2cf0c8f6ce610daa53413def02becb5e.jpg treat image : temp/1758879626_83458_1385917033_792ac47af6c876da68cc5138c522c228.jpg treat image : temp/1758879626_83458_1385917028_792a4b8384220c61ff9ac3d631c90525.jpg treat image : temp/1758879626_83458_1385916960_502236ef52b15522d754ba04261b6a5a.jpg treat image : temp/1758879626_83458_1385916959_baadcfe4f056def10cfcd1a6792738a6.jpg treat image : temp/1758879626_83458_1385914296_87917bfbf79d1a441e4f15350869f773.jpg treat image : temp/1758879626_83458_1385914264_d79e10b5eb1c64ac58256d49dc1ff5e5.jpg treat image : temp/1758879626_83458_1385914122_88fcbbe9eabc17c9021648a7b0818333.jpg treat image : temp/1758879626_83458_1385914118_0a4c44da47fffb92667e02a7f0788532.jpg treat image : temp/1758879626_83458_1385914115_3c7302ea7ea8677ec132999a877a8b8f.jpg treat image : temp/1758879626_83458_1385914109_662809647568715131f7e41be6cc5ba4.jpg treat image : temp/1758879626_83458_1385914108_6975dff1f60dbf420d1c596e6fe3ae0d.jpg treat image : temp/1758879626_83458_1385914105_386102d8f00c37b800f2d06233429352.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 15 time used for this insertion : 0.01383209228515625 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 15 time used for this insertion : 0.013196468353271484 save missing photos in datou_result : time spend for datou_step_exec : 25.178903818130493 time spend to save output : 0.14928078651428223 total time spend for step 2 : 25.328184604644775 step3:blur_detection Fri Sep 26 11:42:38 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1758879626_83458_1385917724_285359095633e41fa610a45583900295.jpg resize: (3040, 4056) 1385917724 0.24040326405463058 treat image : temp/1758879626_83458_1385917717_9cc153cbddc93da0a48d94c3edf0fea8.jpg resize: (3040, 4056) 1385917717 -0.22623397330446895 treat image : temp/1758879626_83458_1385917713_2cf0c8f6ce610daa53413def02becb5e.jpg resize: (3040, 4056) 1385917713 0.1643390090455636 treat image : temp/1758879626_83458_1385917033_792ac47af6c876da68cc5138c522c228.jpg resize: (3040, 4056) 1385917033 0.05502379523730346 treat image : temp/1758879626_83458_1385917028_792a4b8384220c61ff9ac3d631c90525.jpg resize: (3040, 4056) 1385917028 0.32017814490672764 treat image : temp/1758879626_83458_1385916960_502236ef52b15522d754ba04261b6a5a.jpg resize: (3040, 4056) 1385916960 -2.5671850680881705 treat image : temp/1758879626_83458_1385916959_baadcfe4f056def10cfcd1a6792738a6.jpg resize: (3040, 4056) 1385916959 -0.09327647244606813 treat image : temp/1758879626_83458_1385914296_87917bfbf79d1a441e4f15350869f773.jpg resize: (3040, 4056) 1385914296 -2.5671850680881705 treat image : temp/1758879626_83458_1385914264_d79e10b5eb1c64ac58256d49dc1ff5e5.jpg resize: (3040, 4056) 1385914264 -0.09327647244606813 treat image : temp/1758879626_83458_1385914122_88fcbbe9eabc17c9021648a7b0818333.jpg resize: (3040, 4056) 1385914122 0.5603198684518266 treat image : temp/1758879626_83458_1385914118_0a4c44da47fffb92667e02a7f0788532.jpg resize: (3040, 4056) 1385914118 0.5345923916377042 treat image : temp/1758879626_83458_1385914115_3c7302ea7ea8677ec132999a877a8b8f.jpg resize: (3040, 4056) 1385914115 0.3362899798009341 treat image : temp/1758879626_83458_1385914109_662809647568715131f7e41be6cc5ba4.jpg resize: (3040, 4056) 1385914109 0.29513063349902513 treat image : temp/1758879626_83458_1385914108_6975dff1f60dbf420d1c596e6fe3ae0d.jpg resize: (3040, 4056) 1385914108 0.29371167790296515 treat image : temp/1758879626_83458_1385914105_386102d8f00c37b800f2d06233429352.jpg resize: (3040, 4056) 1385914105 0.441736436964093 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 15 time used for this insertion : 0.013014078140258789 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 15 time used for this insertion : 0.012073278427124023 save missing photos in datou_result : time spend for datou_step_exec : 97.92009544372559 time spend to save output : 0.05491304397583008 total time spend for step 3 : 97.97500848770142 step4:crop_condition Fri Sep 26 11:44:16 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 15 ! batch 1 Loaded 129 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 17 About to insert : list_path_to_insert length 17 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 31 About to insert : list_path_to_insert length 31 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 21 About to insert : list_path_to_insert length 21 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1385917724, 1385917717, 1385917713, 1385917033, 1385917028, 1385916960, 1385916959, 1385914296, 1385914264, 1385914122, 1385914118, 1385914115, 1385914109, 1385914108, 1385914105] Looping around the photos to save general results len do output : 88 /-3976672475Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672487Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672488Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672510Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672511Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672512Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672521Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672529Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672530Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672531Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672540Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672564Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672579Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672582Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672586Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672594Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672598Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672506Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672525Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672544Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672549Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672581Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672592Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672485Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672486Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672490Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672548Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672553Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672570Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672571Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672572Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672479Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672493Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672498Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672500Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672501Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672503Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672504Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672505Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672513Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672514Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672517Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672518Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672532Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672533Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672536Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672538Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672545Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672550Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672551Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672554Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672556Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672559Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672560Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672561Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672567Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672568Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672580Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672589Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672591Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672593Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672599Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672474Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672476Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672480Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672495Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672496Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672497Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672499Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672507Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672526Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672546Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672547Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672552Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672557Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672563Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672565Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672569Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672578Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672583Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672585Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672595Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672596Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672509Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672516Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672528Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672535Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976672566Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917724', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917717', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917713', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917033', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917028', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385916960', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385916959', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914296', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914264', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914122', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914118', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914115', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914109', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914108', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914105', None, None, None, None, None, '3772638') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 279 time used for this insertion : 0.0363001823425293 save_final save missing photos in datou_result : time spend for datou_step_exec : 81.91435289382935 time spend to save output : 0.059484243392944336 total time spend for step 4 : 81.97383713722229 step5:thcl Fri Sep 26 11:45:38 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 1 we are using the classfication for multi_thcl [2456] time to import caffe and check if the image exist : 0.018837928771972656 time to convert the images to numpy array : 0.0726468563079834 time to import caffe and check if the image exist : 0.021859407424926758 time to convert the images to numpy array : 0.7775037288665771 time to import caffe and check if the image exist : 0.02574896812438965 time to convert the images to numpy array : 0.8923389911651611 time to import caffe and check if the image exist : 0.01850581169128418 time to convert the images to numpy array : 0.9044501781463623 time to import caffe and check if the image exist : 0.014992713928222656 time to convert the images to numpy array : 1.134629249572754 time to import caffe and check if the image exist : 0.02447652816772461 time to convert the images to numpy array : 1.1260817050933838 time to import caffe and check if the image exist : 0.017544031143188477 time to convert the images to numpy array : 1.1355502605438232 time to import caffe and check if the image exist : 0.018849849700927734 time to convert the images to numpy array : 1.1895523071289062 time to import caffe and check if the image exist : 0.020667552947998047 time to convert the images to numpy array : 1.1981399059295654 time to import caffe and check if the image exist : 0.011343717575073242 time to convert the images to numpy array : 1.2537758350372314 total time to convert the images to numpy array : 1.6632914543151855 list photo_ids error: [] list photo_ids correct : [-3976672476, -3976672480, -3976672495, -3976672496, -3976672497, -3976672499, -3976672507, -3976672526, -3976672546, -3976672547, -3976672552, -3976672557, -3976672563, -3976672565, -3976672569, -3976672578, -3976672583, -3976672585, -3976672503, -3976672504, -3976672505, -3976672513, -3976672514, -3976672517, -3976672518, -3976672532, -3976672533, -3976672536, -3976672538, -3976672545, -3976672550, -3976672551, -3976672554, -3976672556, -3976672559, -3976672560, -3976672595, -3976672596, -3976672509, -3976672516, -3976672528, -3976672535, -3976672566, -3976672531, -3976672540, -3976672564, -3976672579, -3976672582, -3976672586, -3976672594, -3976672598, -3976672506, -3976672561, -3976672567, -3976672568, -3976672580, -3976672589, -3976672591, -3976672593, -3976672599, -3976672474, -3976672553, -3976672570, -3976672571, -3976672572, -3976672479, -3976672493, -3976672498, -3976672500, -3976672501, -3976672525, -3976672544, -3976672549, -3976672581, -3976672592, -3976672485, -3976672486, -3976672490, -3976672548, -3976672475, -3976672487, -3976672488, -3976672510, -3976672511, -3976672512, -3976672521, -3976672529, -3976672530] number of photos to traite : 88 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 7034 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/:/home/admin/workarea/git/apy/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 6891 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.2210383415222168 time used to do the prediction : 0.24743080139160156 we don't save the descriptors for this thcl 2456 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 1.9073486328125e-05 save missing photos in datou_result : time spend for datou_step_exec : 8.655286073684692 time spend to save output : 0.0002777576446533203 total time spend for step 5 : 8.655563831329346 step6:tfhub_classification2 Fri Sep 26 11:45:46 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou_step TFHub with tf2 ! we are using the classfication for only one thcl 3501 begin to check gpu status inside check gpu memory 2025-09-26 11:45:50.915881: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-26 11:45:50.917443: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:45:50.917542: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:45:50.917620: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:45:50.934968: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:45:50.935128: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:45:50.941383: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:45:50.943578: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:45:50.958189: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:45:50.959967: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:45:50.960915: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-26 11:45:50.992710: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-26 11:45:50.994458: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f4120000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-26 11:45:50.994488: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-26 11:45:51.000039: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x4a34d9e0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-26 11:45:51.000072: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-26 11:45:51.001350: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:45:51.001422: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:45:51.001450: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:45:51.001526: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:45:51.001557: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:45:51.001594: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:45:51.001635: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:45:51.001677: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:45:51.003770: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:45:51.004407: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:45:51.004470: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-26 11:45:51.004487: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-26 11:45:51.004501: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-26 11:45:51.006164: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3096 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) l 3637 free memory gpu now : 6905 max_wait_temp : 1 max_wait : 5 1 Physical GPUs, 1 Logical GPUs tagging for thcl : 3501 To do loadFromThcl(), then load ParamDescType : thcl3501 thcls : [{'id': 3501, 'mtr_user_id': 31, 'name': 'learn_papier_mixte_centre_tf_b4_220223_3', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '9087724,9087727,9087725,9087729,9087726,9087723,9087722,9087720,9087721,9087728', 'photo_hashtag_type': 4539, 'photo_desc_type': 5759, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 3501, 'mtr_user_id': 31, 'name': 'learn_papier_mixte_centre_tf_b4_220223_3', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '9087724,9087727,9087725,9087729,9087726,9087723,9087722,9087720,9087721,9087728', 'photo_hashtag_type': 4539, 'photo_desc_type': 5759, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5759 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5759, 'learn_papier_mixte_centre_tf_b4_220223_3', 1792, 1792, 'learn_papier_mixte_centre_tf_b4_220223_3', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 2, 22, 13, 54, 32), datetime.datetime(2023, 2, 22, 13, 54, 32)) model_name : learn_papier_mixte_centre_tf_b4_220223_3 model_param file didn't exist model_name : learn_papier_mixte_centre_tf_b4_220223_3 model_type : tf_classification2 list file need : ['Confusion_Matrix.png', 'Precision_Recall_Carton_brun.jpg', 'Precision_Recall_Carton_gris.jpg', 'Precision_Recall_Teint_Dans_La_Masse.jpg', 'Precision_Recall_autre_refus.jpg', 'Precision_Recall_cartonnette.jpg', 'Precision_Recall_environnement.jpg', 'Precision_Recall_kraft.jpg', 'Precision_Recall_metal.jpg', 'Precision_Recall_papier.jpg', 'Precision_Recall_plastique.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file exist in s3 : ['Confusion_Matrix.png', 'Precision_Recall_Carton_brun.jpg', 'Precision_Recall_Carton_gris.jpg', 'Precision_Recall_Teint_Dans_La_Masse.jpg', 'Precision_Recall_autre_refus.jpg', 'Precision_Recall_cartonnette.jpg', 'Precision_Recall_environnement.jpg', 'Precision_Recall_kraft.jpg', 'Precision_Recall_metal.jpg', 'Precision_Recall_papier.jpg', 'Precision_Recall_plastique.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file manque in s3 : [] local folder : /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3 /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Confusion_Matrix.png size_local : 121269 size in s3 : 121269 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:17 Confusion_Matrix.png already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_Carton_brun.jpg size_local : 76104 size in s3 : 76104 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_Carton_brun.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_Carton_gris.jpg size_local : 82912 size in s3 : 82912 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_Carton_gris.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_Teint_Dans_La_Masse.jpg size_local : 75423 size in s3 : 75423 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:25 Precision_Recall_Teint_Dans_La_Masse.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_autre_refus.jpg size_local : 78883 size in s3 : 78883 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_autre_refus.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_cartonnette.jpg size_local : 81791 size in s3 : 81791 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:30 Precision_Recall_cartonnette.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_environnement.jpg size_local : 59936 size in s3 : 59936 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:17 Precision_Recall_environnement.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_kraft.jpg size_local : 80280 size in s3 : 80280 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:25 Precision_Recall_kraft.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_metal.jpg size_local : 77736 size in s3 : 77736 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_metal.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_papier.jpg size_local : 69041 size in s3 : 69041 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_papier.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_plastique.jpg size_local : 70770 size in s3 : 70770 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:17 Precision_Recall_plastique.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Result_Summary.txt size_local : 1092 size in s3 : 1092 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Result_Summary.txt already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/checkpoint size_local : 99 size in s3 : 99 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 checkpoint already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_checkpoint.ckpt.data-00000-of-00002 size_local : 429801 size in s3 : 429801 create time local : 2023-03-03 12:26:16 create time in s3 : 2023-02-22 12:54:30 model_checkpoint.ckpt.data-00000-of-00002 already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_checkpoint.ckpt.data-00001-of-00002 size_local : 141033188 size in s3 : 141033188 create time local : 2023-03-03 12:26:17 create time in s3 : 2023-02-22 12:54:25 model_checkpoint.ckpt.data-00001-of-00002 already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_checkpoint.ckpt.index size_local : 85618 size in s3 : 85618 create time local : 2023-03-03 12:26:17 create time in s3 : 2023-02-22 12:54:30 model_checkpoint.ckpt.index already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_weights.h5 size_local : 71289448 size in s3 : 71289448 create time local : 2023-03-03 12:26:18 create time in s3 : 2023-02-22 12:54:17 model_weights.h5 already exist and didn't need to update desc size : 1792 Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= module (KerasLayer) (None, 1792) 17673816 _________________________________________________________________ learn_papier_mixte_centre_tf (None, 10) 17930 ================================================================= Total params: 17,691,746 Trainable params: 17,930 Non-trainable params: 17,673,816 _________________________________________________________________ Loading Weights... time used to create the model : 18.469542026519775 time used to load_weights : 0.29531073570251465 0it [00:00, ?it/s] 88it [00:00, 1874.12it/s]2025-09-26 11:46:12.985206: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:46:12.991266: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 temp/1758879626_83458_1385917724_285359095633e41fa610a45583900295_rle_crop_3976672475_0.png temp/1758879626_83458_1385917713_2cf0c8f6ce610daa53413def02becb5e_rle_crop_3976672487_0.png temp/1758879626_83458_1385917713_2cf0c8f6ce610daa53413def02becb5e_rle_crop_3976672488_0.png temp/1758879626_83458_1385916960_502236ef52b15522d754ba04261b6a5a_rle_crop_3976672510_0.png temp/1758879626_83458_1385916960_502236ef52b15522d754ba04261b6a5a_rle_crop_3976672511_0.png temp/1758879626_83458_1385916960_502236ef52b15522d754ba04261b6a5a_rle_crop_3976672512_0.png temp/1758879626_83458_1385916959_baadcfe4f056def10cfcd1a6792738a6_rle_crop_3976672521_0.png temp/1758879626_83458_1385914296_87917bfbf79d1a441e4f15350869f773_rle_crop_3976672529_0.png temp/1758879626_83458_1385914296_87917bfbf79d1a441e4f15350869f773_rle_crop_3976672530_0.png temp/1758879626_83458_1385914296_87917bfbf79d1a441e4f15350869f773_rle_crop_3976672531_0.png temp/1758879626_83458_1385914264_d79e10b5eb1c64ac58256d49dc1ff5e5_rle_crop_3976672540_0.png temp/1758879626_83458_1385914115_3c7302ea7ea8677ec132999a877a8b8f_rle_crop_3976672564_0.png temp/1758879626_83458_1385914109_662809647568715131f7e41be6cc5ba4_rle_crop_3976672579_0.png temp/1758879626_83458_1385914109_662809647568715131f7e41be6cc5ba4_rle_crop_3976672582_0.png temp/1758879626_83458_1385914108_6975dff1f60dbf420d1c596e6fe3ae0d_rle_crop_3976672586_0.png temp/1758879626_83458_1385914105_386102d8f00c37b800f2d06233429352_rle_crop_3976672594_0.png temp/1758879626_83458_1385914105_386102d8f00c37b800f2d06233429352_rle_crop_3976672598_0.png temp/1758879626_83458_1385917028_792a4b8384220c61ff9ac3d631c90525_rle_crop_3976672506_0.png temp/1758879626_83458_1385916959_baadcfe4f056def10cfcd1a6792738a6_rle_crop_3976672525_0.png temp/1758879626_83458_1385914264_d79e10b5eb1c64ac58256d49dc1ff5e5_rle_crop_3976672544_0.png temp/1758879626_83458_1385914122_88fcbbe9eabc17c9021648a7b0818333_rle_crop_3976672549_0.png temp/1758879626_83458_1385914109_662809647568715131f7e41be6cc5ba4_rle_crop_3976672581_0.png temp/1758879626_83458_1385914108_6975dff1f60dbf420d1c596e6fe3ae0d_rle_crop_3976672592_0.png temp/1758879626_83458_1385917713_2cf0c8f6ce610daa53413def02becb5e_rle_crop_3976672485_0.png temp/1758879626_83458_1385917713_2cf0c8f6ce610daa53413def02becb5e_rle_crop_3976672486_0.png temp/1758879626_83458_1385917713_2cf0c8f6ce610daa53413def02becb5e_rle_crop_3976672490_0.png temp/1758879626_83458_1385914122_88fcbbe9eabc17c9021648a7b0818333_rle_crop_3976672548_0.png temp/1758879626_83458_1385914122_88fcbbe9eabc17c9021648a7b0818333_rle_crop_3976672553_0.png temp/1758879626_83458_1385914115_3c7302ea7ea8677ec132999a877a8b8f_rle_crop_3976672570_0.png temp/1758879626_83458_1385914115_3c7302ea7ea8677ec132999a877a8b8f_rle_crop_3976672571_0.png temp/1758879626_83458_1385914115_3c7302ea7ea8677ec132999a877a8b8f_rle_crop_3976672572_0.png temp/1758879626_83458_1385917724_285359095633e41fa610a45583900295_rle_crop_3976672479_0.png temp/1758879626_83458_1385917713_2cf0c8f6ce610daa53413def02becb5e_rle_crop_3976672493_0.png temp/1758879626_83458_1385917033_792ac47af6c876da68cc5138c522c228_rle_crop_3976672498_0.png temp/1758879626_83458_1385917033_792ac47af6c876da68cc5138c522c228_rle_crop_3976672500_0.png temp/1758879626_83458_1385917033_792ac47af6c876da68cc5138c522c228_rle_crop_3976672501_0.png temp/1758879626_83458_1385917028_792a4b8384220c61ff9ac3d631c90525_rle_crop_3976672503_0.png temp/1758879626_83458_1385917028_792a4b8384220c61ff9ac3d631c90525_rle_crop_3976672504_0.png temp/1758879626_83458_1385917028_792a4b8384220c61ff9ac3d631c90525_rle_crop_3976672505_0.png temp/1758879626_83458_1385916960_502236ef52b15522d754ba04261b6a5a_rle_crop_3976672513_0.png temp/1758879626_83458_1385916960_502236ef52b15522d754ba04261b6a5a_rle_crop_3976672514_0.png temp/1758879626_83458_1385916960_502236ef52b15522d754ba04261b6a5a_rle_crop_3976672517_0.png temp/1758879626_83458_1385916960_502236ef52b15522d754ba04261b6a5a_rle_crop_3976672518_0.png temp/1758879626_83458_1385914296_87917bfbf79d1a441e4f15350869f773_rle_crop_3976672532_0.png temp/1758879626_83458_1385914296_87917bfbf79d1a441e4f15350869f773_rle_crop_3976672533_0.png temp/1758879626_83458_1385914296_87917bfbf79d1a441e4f15350869f773_rle_crop_3976672536_0.png temp/1758879626_83458_1385914296_87917bfbf79d1a441e4f15350869f773_rle_crop_3976672538_0.png temp/1758879626_83458_1385914122_88fcbbe9eabc17c9021648a7b0818333_rle_crop_3976672545_0.png temp/1758879626_83458_1385914122_88fcbbe9eabc17c9021648a7b0818333_rle_crop_3976672550_0.png temp/1758879626_83458_1385914122_88fcbbe9eabc17c9021648a7b0818333_rle_crop_3976672551_0.png temp/1758879626_83458_1385914122_88fcbbe9eabc17c9021648a7b0818333_rle_crop_3976672554_0.png temp/1758879626_83458_1385914122_88fcbbe9eabc17c9021648a7b0818333_rle_crop_3976672556_0.png temp/1758879626_83458_1385914118_0a4c44da47fffb92667e02a7f0788532_rle_crop_3976672559_0.png temp/1758879626_83458_1385914118_0a4c44da47fffb92667e02a7f0788532_rle_crop_3976672560_0.png temp/1758879626_83458_1385914118_0a4c44da47fffb92667e02a7f0788532_rle_crop_3976672561_0.png temp/1758879626_83458_1385914115_3c7302ea7ea8677ec132999a877a8b8f_rle_crop_3976672567_0.png temp/1758879626_83458_1385914115_3c7302ea7ea8677ec132999a877a8b8f_rle_crop_3976672568_0.png temp/1758879626_83458_1385914109_662809647568715131f7e41be6cc5ba4_rle_crop_3976672580_0.png temp/1758879626_83458_1385914108_6975dff1f60dbf420d1c596e6fe3ae0d_rle_crop_3976672589_0.png temp/1758879626_83458_1385914108_6975dff1f60dbf420d1c596e6fe3ae0d_rle_crop_3976672591_0.png temp/1758879626_83458_1385914105_386102d8f00c37b800f2d06233429352_rle_crop_3976672593_0.png temp/1758879626_83458_1385914105_386102d8f00c37b800f2d06233429352_rle_crop_3976672599_0.png temp/1758879626_83458_1385917724_285359095633e41fa610a45583900295_rle_crop_3976672474_0.png temp/1758879626_83458_1385917724_285359095633e41fa610a45583900295_rle_crop_3976672476_0.png temp/1758879626_83458_1385917717_9cc153cbddc93da0a48d94c3edf0fea8_rle_crop_3976672480_0.png temp/1758879626_83458_1385917033_792ac47af6c876da68cc5138c522c228_rle_crop_3976672495_0.png temp/1758879626_83458_1385917033_792ac47af6c876da68cc5138c522c228_rle_crop_3976672496_0.png temp/1758879626_83458_1385917033_792ac47af6c876da68cc5138c522c228_rle_crop_3976672497_0.png temp/1758879626_83458_1385917033_792ac47af6c876da68cc5138c522c228_rle_crop_3976672499_0.png temp/1758879626_83458_1385916960_502236ef52b15522d754ba04261b6a5a_rle_crop_3976672507_0.png temp/1758879626_83458_1385914296_87917bfbf79d1a441e4f15350869f773_rle_crop_3976672526_0.png temp/1758879626_83458_1385914122_88fcbbe9eabc17c9021648a7b0818333_rle_crop_3976672546_0.png temp/1758879626_83458_1385914122_88fcbbe9eabc17c9021648a7b0818333_rle_crop_3976672547_0.png temp/1758879626_83458_1385914122_88fcbbe9eabc17c9021648a7b0818333_rle_crop_3976672552_0.png temp/1758879626_83458_1385914118_0a4c44da47fffb92667e02a7f0788532_rle_crop_3976672557_0.png temp/1758879626_83458_1385914115_3c7302ea7ea8677ec132999a877a8b8f_rle_crop_3976672563_0.png temp/1758879626_83458_1385914115_3c7302ea7ea8677ec132999a877a8b8f_rle_crop_3976672565_0.png temp/1758879626_83458_1385914115_3c7302ea7ea8677ec132999a877a8b8f_rle_crop_3976672569_0.png temp/1758879626_83458_1385914109_662809647568715131f7e41be6cc5ba4_rle_crop_3976672578_0.png temp/1758879626_83458_1385914108_6975dff1f60dbf420d1c596e6fe3ae0d_rle_crop_3976672583_0.png temp/1758879626_83458_1385914108_6975dff1f60dbf420d1c596e6fe3ae0d_rle_crop_3976672585_0.png temp/1758879626_83458_1385914105_386102d8f00c37b800f2d06233429352_rle_crop_3976672595_0.png temp/1758879626_83458_1385914105_386102d8f00c37b800f2d06233429352_rle_crop_3976672596_0.png temp/1758879626_83458_1385916960_502236ef52b15522d754ba04261b6a5a_rle_crop_3976672509_0.png temp/1758879626_83458_1385916960_502236ef52b15522d754ba04261b6a5a_rle_crop_3976672516_0.png temp/1758879626_83458_1385914296_87917bfbf79d1a441e4f15350869f773_rle_crop_3976672528_0.png temp/1758879626_83458_1385914296_87917bfbf79d1a441e4f15350869f773_rle_crop_3976672535_0.png temp/1758879626_83458_1385914115_3c7302ea7ea8677ec132999a877a8b8f_rle_crop_3976672566_0.png Found 88 images belonging to 1 classes. begin to do the prediction : time used to do the prediction : 5.519309997558594 (88,) (88, 10) (88, 1792) shape of features : (88, 1792) shape of new features : (1, 88, 1792) save descriptor for thcl : 3501 time to traite the descriptors : 0.45603346824645996 storage_type for insertDescriptorsMulti : 3 Missing photo l117 : -3976672475 Missing photo l117 : -3976672487 Missing photo l117 : -3976672488 Missing photo l117 : -3976672510 Missing photo l117 : -3976672511 Missing photo l117 : -3976672512 Missing photo l117 : -3976672521 Missing photo l117 : -3976672529 Missing photo l117 : -3976672530 Missing photo l117 : -3976672531 Missing photo l117 : -3976672540 Missing photo l117 : -3976672564 Missing photo l117 : -3976672579 Missing photo l117 : -3976672582 Missing photo l117 : -3976672586 Missing photo l117 : -3976672594 Missing photo l117 : -3976672598 Missing photo l117 : -3976672506 Missing photo l117 : -3976672525 Missing photo l117 : -3976672544 Missing photo l117 : -3976672549 Missing photo l117 : -3976672581 Missing photo l117 : -3976672592 Missing photo l117 : -3976672485 Missing photo l117 : -3976672486 Missing photo l117 : -3976672490 Missing photo l117 : -3976672548 Missing photo l117 : -3976672553 Missing photo l117 : -3976672570 Missing photo l117 : -3976672571 Missing photo l117 : -3976672572 Missing photo l117 : -3976672479 Missing photo l117 : -3976672493 Missing photo l117 : -3976672498 Missing photo l117 : -3976672500 Missing photo l117 : -3976672501 Missing photo l117 : -3976672503 Missing photo l117 : -3976672504 Missing photo l117 : -3976672505 Missing photo l117 : -3976672513 Missing photo l117 : -3976672514 Missing photo l117 : -3976672517 Missing photo l117 : -3976672518 Missing photo l117 : -3976672532 Missing photo l117 : -3976672533 Missing photo l117 : -3976672536 Missing photo l117 : -3976672538 Missing photo l117 : -3976672545 Missing photo l117 : -3976672550 Missing photo l117 : -3976672551 Missing photo l117 : -3976672554 Missing photo l117 : -3976672556 Missing photo l117 : -3976672559 Missing photo l117 : -3976672560 Missing photo l117 : -3976672561 Missing photo l117 : -3976672567 Missing photo l117 : -3976672568 Missing photo l117 : -3976672580 Missing photo l117 : -3976672589 Missing photo l117 : -3976672591 Missing photo l117 : -3976672593 Missing photo l117 : -3976672599 Missing photo l117 : -3976672474 Missing photo l117 : -3976672476 Missing photo l117 : -3976672480 Missing photo l117 : -3976672495 Missing photo l117 : -3976672496 Missing photo l117 : -3976672497 Missing photo l117 : -3976672499 Missing photo l117 : -3976672507 Missing photo l117 : -3976672526 Missing photo l117 : -3976672546 Missing photo l117 : -3976672547 Missing photo l117 : -3976672552 Missing photo l117 : -3976672557 Missing photo l117 : -3976672563 Missing photo l117 : -3976672565 Missing photo l117 : -3976672569 Missing photo l117 : -3976672578 Missing photo l117 : -3976672583 Missing photo l117 : -3976672585 Missing photo l117 : -3976672595 Missing photo l117 : -3976672596 Missing photo l117 : -3976672509 Missing photo l117 : -3976672516 Missing photo l117 : -3976672528 Missing photo l117 : -3976672535 Missing photo l117 : -3976672566 To insert : -3976672475 Missing photo l134 : -3976672475 To insert : -3976672487 Missing photo l134 : -3976672487 To insert : -3976672488 Missing photo l134 : -3976672488 To insert : -3976672510 Missing photo l134 : -3976672510 To insert : -3976672511 Missing photo l134 : -3976672511 To insert : -3976672512 Missing photo l134 : -3976672512 To insert : -3976672521 Missing photo l134 : -3976672521 To insert : -3976672529 Missing photo l134 : -3976672529 To insert : -3976672530 Missing photo l134 : -3976672530 To insert : -3976672531 Missing photo l134 : -3976672531 To insert : -3976672540 Missing photo l134 : -3976672540 To insert : -3976672564 Missing photo l134 : -3976672564 To insert : -3976672579 Missing photo l134 : -3976672579 To insert : -3976672582 Missing photo l134 : -3976672582 To insert : -3976672586 Missing photo l134 : -3976672586 To insert : -3976672594 Missing photo l134 : -3976672594 To insert : -3976672598 Missing photo l134 : -3976672598 To insert : -3976672506 Missing photo l134 : -3976672506 To insert : -3976672525 Missing photo l134 : -3976672525 To insert : -3976672544 Missing photo l134 : -3976672544 To insert : -3976672549 Missing photo l134 : -3976672549 To insert : -3976672581 Missing photo l134 : -3976672581 To insert : -3976672592 Missing photo l134 : -3976672592 To insert : -3976672485 Missing photo l134 : -3976672485 To insert : -3976672486 Missing photo l134 : -3976672486 To insert : -3976672490 Missing photo l134 : -3976672490 To insert : -3976672548 Missing photo l134 : -3976672548 To insert : -3976672553 Missing photo l134 : -3976672553 To insert : -3976672570 Missing photo l134 : -3976672570 To insert : -3976672571 Missing photo l134 : -3976672571 To insert : -3976672572 Missing photo l134 : -3976672572 To insert : -3976672479 Missing photo l134 : -3976672479 To insert : -3976672493 Missing photo l134 : -3976672493 To insert : -3976672498 Missing photo l134 : -3976672498 To insert : -3976672500 Missing photo l134 : -3976672500 To insert : -3976672501 Missing photo l134 : -3976672501 To insert : -3976672503 Missing photo l134 : -3976672503 To insert : -3976672504 Missing photo l134 : -3976672504 To insert : -3976672505 Missing photo l134 : -3976672505 To insert : -3976672513 Missing photo l134 : -3976672513 To insert : -3976672514 Missing photo l134 : -3976672514 To insert : -3976672517 Missing photo l134 : -3976672517 To insert : -3976672518 Missing photo l134 : -3976672518 To insert : -3976672532 Missing photo l134 : -3976672532 To insert : -3976672533 Missing photo l134 : -3976672533 To insert : -3976672536 Missing photo l134 : -3976672536 To insert : -3976672538 Missing photo l134 : -3976672538 To insert : -3976672545 Missing photo l134 : -3976672545 To insert : -3976672550 Missing photo l134 : -3976672550 To insert : -3976672551 Missing photo l134 : -3976672551 To insert : -3976672554 Missing photo l134 : -3976672554 To insert : -3976672556 Missing photo l134 : -3976672556 To insert : -3976672559 Missing photo l134 : -3976672559 To insert : -3976672560 Missing photo l134 : -3976672560 To insert : -3976672561 Missing photo l134 : -3976672561 To insert : -3976672567 Missing photo l134 : -3976672567 To insert : -3976672568 Missing photo l134 : -3976672568 To insert : -3976672580 Missing photo l134 : -3976672580 To insert : -3976672589 Missing photo l134 : -3976672589 To insert : -3976672591 Missing photo l134 : -3976672591 To insert : -3976672593 Missing photo l134 : -3976672593 To insert : -3976672599 Missing photo l134 : -3976672599 To insert : -3976672474 Missing photo l134 : -3976672474 To insert : -3976672476 Missing photo l134 : -3976672476 To insert : -3976672480 Missing photo l134 : -3976672480 To insert : -3976672495 Missing photo l134 : -3976672495 To insert : -3976672496 Missing photo l134 : -3976672496 To insert : -3976672497 Missing photo l134 : -3976672497 To insert : -3976672499 Missing photo l134 : -3976672499 To insert : -3976672507 Missing photo l134 : -3976672507 To insert : -3976672526 Missing photo l134 : -3976672526 To insert : -3976672546 Missing photo l134 : -3976672546 To insert : -3976672547 Missing photo l134 : -3976672547 To insert : -3976672552 Missing photo l134 : -3976672552 To insert : -3976672557 Missing photo l134 : -3976672557 To insert : -3976672563 Missing photo l134 : -3976672563 To insert : -3976672565 Missing photo l134 : -3976672565 To insert : -3976672569 Missing photo l134 : -3976672569 To insert : -3976672578 Missing photo l134 : -3976672578 To insert : -3976672583 Missing photo l134 : -3976672583 To insert : -3976672585 Missing photo l134 : -3976672585 To insert : -3976672595 Missing photo l134 : -3976672595 To insert : -3976672596 Missing photo l134 : -3976672596 To insert : -3976672509 Missing photo l134 : -3976672509 To insert : -3976672516 Missing photo l134 : -3976672516 To insert : -3976672528 Missing photo l134 : -3976672528 To insert : -3976672535 Missing photo l134 : -3976672535 To insert : -3976672566 Missing photo l134 : -3976672566 time to insert the descriptors : 16.537484884262085 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : tfhub_classification2 we use saveGeneral [1385917724, 1385917717, 1385917713, 1385917033, 1385917028, 1385916960, 1385916959, 1385914296, 1385914264, 1385914122, 1385914118, 1385914115, 1385914109, 1385914108, 1385914105] Looping around the photos to save general results len do output : 88 /-3976672475Didn't retrieve data . /-3976672487Didn't retrieve data . /-3976672488Didn't retrieve data . /-3976672510Didn't retrieve data . /-3976672511Didn't retrieve data . /-3976672512Didn't retrieve data . /-3976672521Didn't retrieve data . /-3976672529Didn't retrieve data . /-3976672530Didn't retrieve data . /-3976672531Didn't retrieve data . /-3976672540Didn't retrieve data . /-3976672564Didn't retrieve data . /-3976672579Didn't retrieve data . /-3976672582Didn't retrieve data . /-3976672586Didn't retrieve data . /-3976672594Didn't retrieve data . /-3976672598Didn't retrieve data . /-3976672506Didn't retrieve data . /-3976672525Didn't retrieve data . /-3976672544Didn't retrieve data . /-3976672549Didn't retrieve data . /-3976672581Didn't retrieve data . /-3976672592Didn't retrieve data . /-3976672485Didn't retrieve data . /-3976672486Didn't retrieve data . /-3976672490Didn't retrieve data . /-3976672548Didn't retrieve data . /-3976672553Didn't retrieve data . /-3976672570Didn't retrieve data . /-3976672571Didn't retrieve data . /-3976672572Didn't retrieve data . /-3976672479Didn't retrieve data . /-3976672493Didn't retrieve data . /-3976672498Didn't retrieve data . /-3976672500Didn't retrieve data . /-3976672501Didn't retrieve data . /-3976672503Didn't retrieve data . /-3976672504Didn't retrieve data . /-3976672505Didn't retrieve data . /-3976672513Didn't retrieve data . /-3976672514Didn't retrieve data . /-3976672517Didn't retrieve data . /-3976672518Didn't retrieve data . /-3976672532Didn't retrieve data . /-3976672533Didn't retrieve data . /-3976672536Didn't retrieve data . /-3976672538Didn't retrieve data . /-3976672545Didn't retrieve data . /-3976672550Didn't retrieve data . /-3976672551Didn't retrieve data . /-3976672554Didn't retrieve data . /-3976672556Didn't retrieve data . /-3976672559Didn't retrieve data . /-3976672560Didn't retrieve data . /-3976672561Didn't retrieve data . /-3976672567Didn't retrieve data . /-3976672568Didn't retrieve data . /-3976672580Didn't retrieve data . /-3976672589Didn't retrieve data . /-3976672591Didn't retrieve data . /-3976672593Didn't retrieve data . /-3976672599Didn't retrieve data . /-3976672474Didn't retrieve data . /-3976672476Didn't retrieve data . /-3976672480Didn't retrieve data . /-3976672495Didn't retrieve data . /-3976672496Didn't retrieve data . /-3976672497Didn't retrieve data . /-3976672499Didn't retrieve data . /-3976672507Didn't retrieve data . /-3976672526Didn't retrieve data . /-3976672546Didn't retrieve data . /-3976672547Didn't retrieve data . /-3976672552Didn't retrieve data . /-3976672557Didn't retrieve data . /-3976672563Didn't retrieve data . /-3976672565Didn't retrieve data . /-3976672569Didn't retrieve data . /-3976672578Didn't retrieve data . /-3976672583Didn't retrieve data . /-3976672585Didn't retrieve data . /-3976672595Didn't retrieve data . /-3976672596Didn't retrieve data . /-3976672509Didn't retrieve data . /-3976672516Didn't retrieve data . /-3976672528Didn't retrieve data . /-3976672535Didn't retrieve data . /-3976672566Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917724', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917717', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917713', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917033', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917028', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385916960', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385916959', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914296', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914264', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914122', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914118', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914115', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914109', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914108', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914105', None, None, None, None, None, '3772638') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 103 time used for this insertion : 0.029767274856567383 save_final save missing photos in datou_result : time spend for datou_step_exec : 48.997602701187134 time spend to save output : 0.03144431114196777 total time spend for step 6 : 49.0290470123291 step7:merge_mask_thcl_custom Fri Sep 26 11:46:35 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 129 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1385917724, 1385917717, 1385917713, 1385917033, 1385917028, 1385916960, 1385916959, 1385914296, 1385914264, 1385914122, 1385914118, 1385914115, 1385914109, 1385914108, 1385914105] Looping around the photos to save general results len do output : 15 /1385917724Didn't retrieve data .Didn't retrieve data . /1385917717Didn't retrieve data .Didn't retrieve data . /1385917713Didn't retrieve data .Didn't retrieve data . /1385917033Didn't retrieve data .Didn't retrieve data . /1385917028Didn't retrieve data .Didn't retrieve data . /1385916960Didn't retrieve data .Didn't retrieve data . /1385916959Didn't retrieve data .Didn't retrieve data . /1385914296Didn't retrieve data .Didn't retrieve data . /1385914264Didn't retrieve data .Didn't retrieve data . /1385914122Didn't retrieve data .Didn't retrieve data . /1385914118Didn't retrieve data .Didn't retrieve data . /1385914115Didn't retrieve data .Didn't retrieve data . /1385914109Didn't retrieve data .Didn't retrieve data . /1385914108Didn't retrieve data .Didn't retrieve data . /1385914105Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917724', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917717', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917713', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917033', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917028', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385916960', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385916959', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914296', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914264', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914122', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914118', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914115', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914109', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914108', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914105', None, None, None, None, None, '3772638') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 45 time used for this insertion : 0.012830018997192383 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.06250977516174316 time spend to save output : 0.013558387756347656 total time spend for step 7 : 0.07606816291809082 step8:rle_unique_nms_with_priority Fri Sep 26 11:46:35 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 5 nb_hashtags : 3 time to prepare the origin masks : 6.956450700759888 time for calcul the mask position with numpy : 0.6834897994995117 nb_pixel_total : 11626085 time to create 1 rle with new method : 0.7596569061279297 time for calcul the mask position with numpy : 0.037334442138671875 nb_pixel_total : 438758 time to create 1 rle with new method : 0.964956521987915 time for calcul the mask position with numpy : 0.03425407409667969 nb_pixel_total : 36759 time to create 1 rle with old method : 0.03897237777709961 time for calcul the mask position with numpy : 0.036461591720581055 nb_pixel_total : 175683 time to create 1 rle with new method : 0.8519530296325684 time for calcul the mask position with numpy : 0.03524160385131836 nb_pixel_total : 4433 time to create 1 rle with old method : 0.004869937896728516 time for calcul the mask position with numpy : 0.03534102439880371 nb_pixel_total : 48522 time to create 1 rle with old method : 0.052051544189453125 create new chi : 3.67641282081604 time to delete rle : 0.016003847122192383 batch 1 Loaded 6 chid ids of type : 3726 Number RLEs to save : 7212 TO DO : save crop sub photo not yet done ! save time : 0.4029355049133301 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 5.390194416046143 time for calcul the mask position with numpy : 0.6750931739807129 nb_pixel_total : 9515670 time to create 1 rle with new method : 0.9218399524688721 time for calcul the mask position with numpy : 0.034424543380737305 nb_pixel_total : 4104 time to create 1 rle with old method : 0.004549264907836914 time for calcul the mask position with numpy : 0.04309368133544922 nb_pixel_total : 1303831 time to create 1 rle with new method : 0.8347923755645752 time for calcul the mask position with numpy : 0.04610180854797363 nb_pixel_total : 1473754 time to create 1 rle with new method : 0.9039173126220703 time for calcul the mask position with numpy : 0.03481793403625488 nb_pixel_total : 32881 time to create 1 rle with old method : 0.03555941581726074 create new chi : 3.6741302013397217 time to delete rle : 0.0013110637664794922 batch 1 Loaded 5 chid ids of type : 3726 Number RLEs to save : 13440 TO DO : save crop sub photo not yet done ! save time : 0.7103524208068848 nb_obj : 8 nb_hashtags : 5 time to prepare the origin masks : 9.712095499038696 time for calcul the mask position with numpy : 0.5239615440368652 nb_pixel_total : 10258179 time to create 1 rle with new method : 0.592200756072998 time for calcul the mask position with numpy : 0.03983783721923828 nb_pixel_total : 292951 time to create 1 rle with new method : 0.7416470050811768 time for calcul the mask position with numpy : 0.03839707374572754 nb_pixel_total : 47184 time to create 1 rle with old method : 0.05113387107849121 time for calcul the mask position with numpy : 0.0369412899017334 nb_pixel_total : 58281 time to create 1 rle with old method : 0.06384611129760742 time for calcul the mask position with numpy : 0.038147687911987305 nb_pixel_total : 16186 time to create 1 rle with old method : 0.019022703170776367 time for calcul the mask position with numpy : 0.06545782089233398 nb_pixel_total : 1505035 time to create 1 rle with new method : 0.7484781742095947 time for calcul the mask position with numpy : 0.06853389739990234 nb_pixel_total : 89643 time to create 1 rle with old method : 0.09687328338623047 time for calcul the mask position with numpy : 0.061292171478271484 nb_pixel_total : 52202 time to create 1 rle with old method : 0.0602259635925293 time for calcul the mask position with numpy : 0.05391812324523926 nb_pixel_total : 10579 time to create 1 rle with old method : 0.011914253234863281 create new chi : 3.46270751953125 time to delete rle : 0.0012996196746826172 batch 1 Loaded 9 chid ids of type : 3726 Number RLEs to save : 12518 TO DO : save crop sub photo not yet done ! save time : 0.6874966621398926 nb_obj : 7 nb_hashtags : 3 time to prepare the origin masks : 7.832175970077515 time for calcul the mask position with numpy : 0.8313498497009277 nb_pixel_total : 11891605 time to create 1 rle with new method : 0.7977802753448486 time for calcul the mask position with numpy : 0.0359952449798584 nb_pixel_total : 113681 time to create 1 rle with old method : 0.1201026439666748 time for calcul the mask position with numpy : 0.035928964614868164 nb_pixel_total : 8600 time to create 1 rle with old method : 0.009420156478881836 time for calcul the mask position with numpy : 0.036086320877075195 nb_pixel_total : 100038 time to create 1 rle with old method : 0.10737490653991699 time for calcul the mask position with numpy : 0.03464841842651367 nb_pixel_total : 27973 time to create 1 rle with old method : 0.030373096466064453 time for calcul the mask position with numpy : 0.03600502014160156 nb_pixel_total : 2500 time to create 1 rle with old method : 0.0029144287109375 time for calcul the mask position with numpy : 0.03480672836303711 nb_pixel_total : 14619 time to create 1 rle with old method : 0.015346527099609375 time for calcul the mask position with numpy : 0.037468671798706055 nb_pixel_total : 171224 time to create 1 rle with new method : 0.719130277633667 create new chi : 3.000004291534424 time to delete rle : 0.0008294582366943359 batch 1 Loaded 8 chid ids of type : 3726 Number RLEs to save : 6271 TO DO : save crop sub photo not yet done ! save time : 0.39835572242736816 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 6.103817462921143 time for calcul the mask position with numpy : 0.766596794128418 nb_pixel_total : 11713953 time to create 1 rle with new method : 1.2910637855529785 time for calcul the mask position with numpy : 0.042145729064941406 nb_pixel_total : 239256 time to create 1 rle with new method : 0.9813263416290283 time for calcul the mask position with numpy : 0.041792869567871094 nb_pixel_total : 31873 time to create 1 rle with old method : 0.040396690368652344 time for calcul the mask position with numpy : 0.05504345893859863 nb_pixel_total : 41301 time to create 1 rle with old method : 0.055260419845581055 time for calcul the mask position with numpy : 0.05089163780212402 nb_pixel_total : 303857 time to create 1 rle with new method : 1.1386854648590088 create new chi : 4.629457235336304 time to delete rle : 0.0010962486267089844 batch 1 Loaded 5 chid ids of type : 3726 Number RLEs to save : 6682 TO DO : save crop sub photo not yet done ! save time : 0.38996386528015137 nb_obj : 11 nb_hashtags : 4 time to prepare the origin masks : 11.395837783813477 time for calcul the mask position with numpy : 0.47397613525390625 nb_pixel_total : 9520179 time to create 1 rle with new method : 0.5628166198730469 time for calcul the mask position with numpy : 0.05854177474975586 nb_pixel_total : 133146 time to create 1 rle with old method : 0.14020276069641113 time for calcul the mask position with numpy : 0.05949282646179199 nb_pixel_total : 46182 time to create 1 rle with old method : 0.04955291748046875 time for calcul the mask position with numpy : 0.05819964408874512 nb_pixel_total : 36631 time to create 1 rle with old method : 0.8449544906616211 time for calcul the mask position with numpy : 0.06509208679199219 nb_pixel_total : 635355 time to create 1 rle with new method : 0.59755539894104 time for calcul the mask position with numpy : 0.05738711357116699 nb_pixel_total : 105572 time to create 1 rle with old method : 0.11249279975891113 time for calcul the mask position with numpy : 0.05438685417175293 nb_pixel_total : 24423 time to create 1 rle with old method : 0.025526046752929688 time for calcul the mask position with numpy : 0.05668210983276367 nb_pixel_total : 84626 time to create 1 rle with old method : 0.08935856819152832 time for calcul the mask position with numpy : 0.05185079574584961 nb_pixel_total : 15722 time to create 1 rle with old method : 0.016382932662963867 time for calcul the mask position with numpy : 0.0347752571105957 nb_pixel_total : 133309 time to create 1 rle with old method : 0.1413099765777588 time for calcul the mask position with numpy : 0.03525352478027344 nb_pixel_total : 16304 time to create 1 rle with old method : 0.017318248748779297 time for calcul the mask position with numpy : 0.051384925842285156 nb_pixel_total : 1578791 time to create 1 rle with new method : 0.5673823356628418 create new chi : 4.3534932136535645 time to delete rle : 0.001558065414428711 batch 1 Loaded 12 chid ids of type : 3726 Number RLEs to save : 15232 TO DO : save crop sub photo not yet done ! save time : 0.8141775131225586 nb_obj : 6 nb_hashtags : 2 time to prepare the origin masks : 5.053502082824707 time for calcul the mask position with numpy : 0.45872044563293457 nb_pixel_total : 10583751 time to create 1 rle with new method : 0.5703856945037842 time for calcul the mask position with numpy : 0.06197309494018555 nb_pixel_total : 42734 time to create 1 rle with old method : 0.04864025115966797 time for calcul the mask position with numpy : 0.035925865173339844 nb_pixel_total : 8505 time to create 1 rle with old method : 0.009595155715942383 time for calcul the mask position with numpy : 0.043821096420288086 nb_pixel_total : 1139079 time to create 1 rle with new method : 0.6925983428955078 time for calcul the mask position with numpy : 0.03826761245727539 nb_pixel_total : 391943 time to create 1 rle with new method : 0.611804723739624 time for calcul the mask position with numpy : 0.03707075119018555 nb_pixel_total : 128407 time to create 1 rle with old method : 0.13602638244628906 time for calcul the mask position with numpy : 0.03988170623779297 nb_pixel_total : 35821 time to create 1 rle with old method : 0.03845381736755371 create new chi : 2.9615421295166016 time to delete rle : 0.001722097396850586 batch 1 Loaded 7 chid ids of type : 3726 Number RLEs to save : 11348 TO DO : save crop sub photo not yet done ! save time : 0.6004886627197266 nb_obj : 11 nb_hashtags : 4 time to prepare the origin masks : 10.27092719078064 time for calcul the mask position with numpy : 0.4433612823486328 nb_pixel_total : 9520183 time to create 1 rle with new method : 1.4256770610809326 time for calcul the mask position with numpy : 0.04519772529602051 nb_pixel_total : 133211 time to create 1 rle with old method : 0.15602588653564453 time for calcul the mask position with numpy : 0.04653620719909668 nb_pixel_total : 46183 time to create 1 rle with old method : 0.055745601654052734 time for calcul the mask position with numpy : 0.0434417724609375 nb_pixel_total : 36644 time to create 1 rle with old method : 0.04298686981201172 time for calcul the mask position with numpy : 0.04611921310424805 nb_pixel_total : 105643 time to create 1 rle with old method : 0.11368393898010254 time for calcul the mask position with numpy : 0.058620452880859375 nb_pixel_total : 635195 time to create 1 rle with new method : 0.8403286933898926 time for calcul the mask position with numpy : 0.04393315315246582 nb_pixel_total : 24420 time to create 1 rle with old method : 0.02753591537475586 time for calcul the mask position with numpy : 0.0531764030456543 nb_pixel_total : 84624 time to create 1 rle with old method : 0.10154891014099121 time for calcul the mask position with numpy : 0.0418701171875 nb_pixel_total : 15715 time to create 1 rle with old method : 0.01639580726623535 time for calcul the mask position with numpy : 0.044818878173828125 nb_pixel_total : 133165 time to create 1 rle with old method : 0.14397454261779785 time for calcul the mask position with numpy : 0.07340145111083984 nb_pixel_total : 16300 time to create 1 rle with old method : 0.017232656478881836 time for calcul the mask position with numpy : 0.09342527389526367 nb_pixel_total : 1578957 time to create 1 rle with new method : 0.7692170143127441 create new chi : 4.9275970458984375 time to delete rle : 0.0018832683563232422 batch 1 Loaded 12 chid ids of type : 3726 Number RLEs to save : 15217 TO DO : save crop sub photo not yet done ! save time : 0.8103952407836914 nb_obj : 6 nb_hashtags : 2 time to prepare the origin masks : 7.089661598205566 time for calcul the mask position with numpy : 0.6945104598999023 nb_pixel_total : 10585373 time to create 1 rle with new method : 1.6571078300476074 time for calcul the mask position with numpy : 0.05000948905944824 nb_pixel_total : 42690 time to create 1 rle with old method : 0.05428481101989746 time for calcul the mask position with numpy : 0.0406644344329834 nb_pixel_total : 8505 time to create 1 rle with old method : 0.013747930526733398 time for calcul the mask position with numpy : 0.04707074165344238 nb_pixel_total : 1137470 time to create 1 rle with new method : 1.818925142288208 time for calcul the mask position with numpy : 0.04744577407836914 nb_pixel_total : 391910 time to create 1 rle with new method : 1.5860095024108887 time for calcul the mask position with numpy : 0.06977343559265137 nb_pixel_total : 128472 time to create 1 rle with old method : 0.15121197700500488 time for calcul the mask position with numpy : 0.06447887420654297 nb_pixel_total : 35820 time to create 1 rle with old method : 0.040064334869384766 create new chi : 6.5134077072143555 time to delete rle : 0.0017528533935546875 batch 1 Loaded 7 chid ids of type : 3726 Number RLEs to save : 11377 TO DO : save crop sub photo not yet done ! save time : 0.6042275428771973 nb_obj : 11 nb_hashtags : 6 time to prepare the origin masks : 17.093343496322632 time for calcul the mask position with numpy : 0.7593035697937012 nb_pixel_total : 11507327 time to create 1 rle with new method : 1.2148854732513428 time for calcul the mask position with numpy : 0.04683804512023926 nb_pixel_total : 180581 time to create 1 rle with new method : 1.1828696727752686 time for calcul the mask position with numpy : 0.047246694564819336 nb_pixel_total : 81404 time to create 1 rle with old method : 0.09731578826904297 time for calcul the mask position with numpy : 0.0588381290435791 nb_pixel_total : 31310 time to create 1 rle with old method : 0.04415726661682129 time for calcul the mask position with numpy : 0.054566383361816406 nb_pixel_total : 12713 time to create 1 rle with old method : 0.017790794372558594 time for calcul the mask position with numpy : 0.054163217544555664 nb_pixel_total : 40220 time to create 1 rle with old method : 0.04598855972290039 time for calcul the mask position with numpy : 0.04379129409790039 nb_pixel_total : 81645 time to create 1 rle with old method : 0.11620283126831055 time for calcul the mask position with numpy : 0.07366085052490234 nb_pixel_total : 8215 time to create 1 rle with old method : 0.009733915328979492 time for calcul the mask position with numpy : 0.08779501914978027 nb_pixel_total : 43839 time to create 1 rle with old method : 0.04928159713745117 time for calcul the mask position with numpy : 0.07539057731628418 nb_pixel_total : 2924 time to create 1 rle with old method : 0.004309415817260742 time for calcul the mask position with numpy : 0.07272934913635254 nb_pixel_total : 124208 time to create 1 rle with old method : 0.13234758377075195 time for calcul the mask position with numpy : 0.07713437080383301 nb_pixel_total : 215854 time to create 1 rle with new method : 0.832486629486084 create new chi : 5.432295799255371 time to delete rle : 0.0014603137969970703 batch 1 Loaded 12 chid ids of type : 3726 Number RLEs to save : 10060 TO DO : save crop sub photo not yet done ! save time : 0.6118285655975342 nb_obj : 5 nb_hashtags : 4 time to prepare the origin masks : 11.764389514923096 time for calcul the mask position with numpy : 1.498749017715454 nb_pixel_total : 11228698 time to create 1 rle with new method : 1.7408819198608398 time for calcul the mask position with numpy : 0.05917000770568848 nb_pixel_total : 159595 time to create 1 rle with new method : 1.205902338027954 time for calcul the mask position with numpy : 0.0631875991821289 nb_pixel_total : 183146 time to create 1 rle with new method : 1.3143894672393799 time for calcul the mask position with numpy : 0.07167267799377441 nb_pixel_total : 65601 time to create 1 rle with old method : 0.08067488670349121 time for calcul the mask position with numpy : 0.04991769790649414 nb_pixel_total : 675982 time to create 1 rle with new method : 0.7985296249389648 time for calcul the mask position with numpy : 0.04324650764465332 nb_pixel_total : 17218 time to create 1 rle with old method : 0.021060705184936523 create new chi : 7.151189088821411 time to delete rle : 0.0013828277587890625 batch 1 Loaded 6 chid ids of type : 3726 Number RLEs to save : 9883 TO DO : save crop sub photo not yet done ! save time : 0.5434484481811523 nb_obj : 12 nb_hashtags : 6 time to prepare the origin masks : 21.93396806716919 time for calcul the mask position with numpy : 0.9292573928833008 nb_pixel_total : 9600065 time to create 1 rle with new method : 1.808218002319336 time for calcul the mask position with numpy : 0.07112836837768555 nb_pixel_total : 16367 time to create 1 rle with old method : 0.01870274543762207 time for calcul the mask position with numpy : 0.07131266593933105 nb_pixel_total : 25737 time to create 1 rle with old method : 0.028139591217041016 time for calcul the mask position with numpy : 0.06952118873596191 nb_pixel_total : 325924 time to create 1 rle with new method : 1.9511921405792236 time for calcul the mask position with numpy : 0.05755972862243652 nb_pixel_total : 932 time to create 1 rle with old method : 0.0012726783752441406 time for calcul the mask position with numpy : 0.05232071876525879 nb_pixel_total : 37675 time to create 1 rle with old method : 0.039969444274902344 time for calcul the mask position with numpy : 0.037081003189086914 nb_pixel_total : 181186 time to create 1 rle with new method : 1.2540555000305176 time for calcul the mask position with numpy : 0.058246612548828125 nb_pixel_total : 1680024 time to create 1 rle with new method : 1.362617015838623 time for calcul the mask position with numpy : 0.03762412071228027 nb_pixel_total : 33129 time to create 1 rle with old method : 0.03598952293395996 time for calcul the mask position with numpy : 0.03736567497253418 nb_pixel_total : 181127 time to create 1 rle with new method : 1.0001928806304932 time for calcul the mask position with numpy : 0.049709320068359375 nb_pixel_total : 11504 time to create 1 rle with old method : 0.013488292694091797 time for calcul the mask position with numpy : 0.04302048683166504 nb_pixel_total : 82194 time to create 1 rle with old method : 0.09933066368103027 time for calcul the mask position with numpy : 0.062225341796875 nb_pixel_total : 154376 time to create 1 rle with new method : 1.5155537128448486 create new chi : 10.99336051940918 time to delete rle : 0.0015897750854492188 batch 1 Loaded 13 chid ids of type : 3726 Number RLEs to save : 15665 TO DO : save crop sub photo not yet done ! save time : 0.9143717288970947 nb_obj : 5 nb_hashtags : 3 time to prepare the origin masks : 9.10240888595581 time for calcul the mask position with numpy : 1.617624282836914 nb_pixel_total : 11965104 time to create 1 rle with new method : 1.449143886566162 time for calcul the mask position with numpy : 0.044567108154296875 nb_pixel_total : 2571 time to create 1 rle with old method : 0.004691600799560547 time for calcul the mask position with numpy : 0.04766201972961426 nb_pixel_total : 33211 time to create 1 rle with old method : 0.0387423038482666 time for calcul the mask position with numpy : 0.03703022003173828 nb_pixel_total : 158444 time to create 1 rle with new method : 2.4067223072052 time for calcul the mask position with numpy : 0.03896069526672363 nb_pixel_total : 59727 time to create 1 rle with old method : 0.07262158393859863 time for calcul the mask position with numpy : 0.06571030616760254 nb_pixel_total : 111183 time to create 1 rle with old method : 0.1426534652709961 create new chi : 6.093961477279663 time to delete rle : 0.0008900165557861328 batch 1 Loaded 6 chid ids of type : 3726 Number RLEs to save : 6081 TO DO : save crop sub photo not yet done ! save time : 0.37760066986083984 nb_obj : 8 nb_hashtags : 3 time to prepare the origin masks : 18.28731369972229 time for calcul the mask position with numpy : 1.616870403289795 nb_pixel_total : 10831772 time to create 1 rle with new method : 1.7262039184570312 time for calcul the mask position with numpy : 0.06624341011047363 nb_pixel_total : 126206 time to create 1 rle with old method : 0.16480612754821777 time for calcul the mask position with numpy : 0.037709951400756836 nb_pixel_total : 33119 time to create 1 rle with old method : 0.03774428367614746 time for calcul the mask position with numpy : 0.04566383361816406 nb_pixel_total : 775084 time to create 1 rle with new method : 1.4411730766296387 time for calcul the mask position with numpy : 0.05590462684631348 nb_pixel_total : 132662 time to create 1 rle with old method : 0.15543699264526367 time for calcul the mask position with numpy : 0.03852391242980957 nb_pixel_total : 125062 time to create 1 rle with old method : 0.13800334930419922 time for calcul the mask position with numpy : 0.04485344886779785 nb_pixel_total : 235136 time to create 1 rle with new method : 1.7061967849731445 time for calcul the mask position with numpy : 0.03765702247619629 nb_pixel_total : 24720 time to create 1 rle with old method : 0.0271451473236084 time for calcul the mask position with numpy : 0.043939828872680664 nb_pixel_total : 46479 time to create 1 rle with old method : 0.053527116775512695 create new chi : 7.6078715324401855 time to delete rle : 0.0013935565948486328 batch 1 Loaded 9 chid ids of type : 3726 Number RLEs to save : 13001 TO DO : save crop sub photo not yet done ! save time : 0.7895431518554688 nb_obj : 8 nb_hashtags : 5 time to prepare the origin masks : 15.88141417503357 time for calcul the mask position with numpy : 1.3818786144256592 nb_pixel_total : 11022586 time to create 1 rle with new method : 1.246664047241211 time for calcul the mask position with numpy : 0.05763983726501465 nb_pixel_total : 43484 time to create 1 rle with old method : 0.04553103446960449 time for calcul the mask position with numpy : 0.0405118465423584 nb_pixel_total : 1029194 time to create 1 rle with new method : 1.0319409370422363 time for calcul the mask position with numpy : 0.03717470169067383 nb_pixel_total : 974 time to create 1 rle with old method : 0.0013935565948486328 time for calcul the mask position with numpy : 0.036529541015625 nb_pixel_total : 126412 time to create 1 rle with old method : 0.14446091651916504 time for calcul the mask position with numpy : 0.041268110275268555 nb_pixel_total : 33170 time to create 1 rle with old method : 0.04076719284057617 time for calcul the mask position with numpy : 0.03893852233886719 nb_pixel_total : 28379 time to create 1 rle with old method : 0.031333208084106445 time for calcul the mask position with numpy : 0.037308454513549805 nb_pixel_total : 5327 time to create 1 rle with old method : 0.00623774528503418 time for calcul the mask position with numpy : 0.04431033134460449 nb_pixel_total : 40714 time to create 1 rle with old method : 0.04506993293762207 create new chi : 4.4121057987213135 time to delete rle : 0.0012350082397460938 batch 1 Loaded 9 chid ids of type : 3726 Number RLEs to save : 11298 TO DO : save crop sub photo not yet done ! save time : 0.643120288848877 map_output_result : {1385917724: (0.0, 'Should be the crop_list due to order', 0.0), 1385917717: (0.0, 'Should be the crop_list due to order', 0.0), 1385917713: (0.0, 'Should be the crop_list due to order', 0.0), 1385917033: (0.0, 'Should be the crop_list due to order', 0.0), 1385917028: (0.0, 'Should be the crop_list due to order', 0.0), 1385916960: (0.0, 'Should be the crop_list due to order', 0.0), 1385916959: (0.0, 'Should be the crop_list due to order', 0.0), 1385914296: (0.0, 'Should be the crop_list due to order', 0.0), 1385914264: (0.0, 'Should be the crop_list due to order', 0.0), 1385914122: (0.0, 'Should be the crop_list due to order', 0.0), 1385914118: (0.0, 'Should be the crop_list due to order', 0.0), 1385914115: (0.0, 'Should be the crop_list due to order', 0.0), 1385914109: (0.0, 'Should be the crop_list due to order', 0.0), 1385914108: (0.0, 'Should be the crop_list due to order', 0.0), 1385914105: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1385917724, 1385917717, 1385917713, 1385917033, 1385917028, 1385916960, 1385916959, 1385914296, 1385914264, 1385914122, 1385914118, 1385914115, 1385914109, 1385914108, 1385914105] Looping around the photos to save general results len do output : 15 /1385917724.Didn't retrieve data . /1385917717.Didn't retrieve data . /1385917713.Didn't retrieve data . /1385917033.Didn't retrieve data . /1385917028.Didn't retrieve data . /1385916960.Didn't retrieve data . /1385916959.Didn't retrieve data . /1385914296.Didn't retrieve data . /1385914264.Didn't retrieve data . /1385914122.Didn't retrieve data . /1385914118.Didn't retrieve data . /1385914115.Didn't retrieve data . /1385914109.Didn't retrieve data . /1385914108.Didn't retrieve data . /1385914105.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917724', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917717', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917713', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917033', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917028', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385916960', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385916959', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914296', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914264', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914122', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914118', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914115', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914109', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914108', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914105', None, None, None, None, None, '3772638') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 45 time used for this insertion : 0.015070438385009766 save_final save missing photos in datou_result : time spend for datou_step_exec : 256.4299795627594 time spend to save output : 0.016838550567626953 total time spend for step 8 : 256.446818113327 step9:crop_condition Fri Sep 26 11:50:52 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 15 ! batch 1 Loaded 126 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! About to upload 7 photos upload in portfolio : 4869462 init cache_photo without model_param we have 7 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758880263_83458 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 7 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.5557901859283447 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! About to upload 8 photos upload in portfolio : 4869462 init cache_photo without model_param we have 8 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758880275_83458 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 8 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.0109333992004395 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 18 About to insert : list_path_to_insert length 18 new photo from crops ! About to upload 18 photos upload in portfolio : 4869462 init cache_photo without model_param we have 18 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758880290_83458 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 18 photos in the portfolio 4869462 time of upload the photos Elapsed time : 5.188416004180908 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1385917724, 1385917717, 1385917713, 1385917033, 1385917028, 1385916960, 1385916959, 1385914296, 1385914264, 1385914122, 1385914118, 1385914115, 1385914109, 1385914108, 1385914105] Looping around the photos to save general results len do output : 33 /1386337408Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337409Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337410Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337412Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337413Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337414Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337416Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337443Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337444Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337445Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337446Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337447Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337448Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337449Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337450Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337460Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337461Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337462Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337463Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337464Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337465Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337466Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337468Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337470Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337472Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337473Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337474Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337475Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337476Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337477Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337478Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337479Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386337480Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917724', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917717', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917713', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917033', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917028', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385916960', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385916959', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914296', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914264', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914122', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914118', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914115', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914109', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914108', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914105', None, None, None, None, None, '3772638') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 114 time used for this insertion : 0.03901243209838867 save_final save missing photos in datou_result : time spend for datou_step_exec : 44.45392322540283 time spend to save output : 0.04547619819641113 total time spend for step 9 : 44.49939942359924 step10:ventilate_hashtags_in_portfolio Fri Sep 26 11:51:36 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 27193435 get user id for portfolio 27193435 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27193435 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton_brun','Teint_Dans_La_Masse','Carton_gris','plastique','flou','metal','cartonnette','environnement','autre_refus','kraft','mal_croppe','papier')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27193435 AND mptpi.`type`=4417 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton_brun','Teint_Dans_La_Masse','Carton_gris','plastique','flou','metal','cartonnette','environnement','autre_refus','kraft','mal_croppe','papier')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://marlene.fotonower.com/velours/27273428,27273429,27273430,27273431,27273432,27273433,27273434,27273435,27273436,27273437,27273438,27273439?tags=autre_refus,flou,papier,mal_croppe,metal,Teint_Dans_La_Masse,plastique,Carton_gris,Carton_brun,kraft,environnement,cartonnette Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1385917724, 1385917717, 1385917713, 1385917033, 1385917028, 1385916960, 1385916959, 1385914296, 1385914264, 1385914122, 1385914118, 1385914115, 1385914109, 1385914108, 1385914105] Looping around the photos to save general results len do output : 1 /27193435. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917724', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917717', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917713', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917033', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917028', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385916960', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385916959', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914296', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914264', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914122', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914118', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914115', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914109', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914108', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914105', None, None, None, None, None, '3772638') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 16 time used for this insertion : 0.01468968391418457 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.265037775039673 time spend to save output : 0.01647162437438965 total time spend for step 10 : 2.2815093994140625 step11:final Fri Sep 26 11:51:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1385917724: ('0.10030687056887812',), 1385917717: ('0.10030687056887812',), 1385917713: ('0.10030687056887812',), 1385917033: ('0.10030687056887812',), 1385917028: ('0.10030687056887812',), 1385916960: ('0.10030687056887812',), 1385916959: ('0.10030687056887812',), 1385914296: ('0.10030687056887812',), 1385914264: ('0.10030687056887812',), 1385914122: ('0.10030687056887812',), 1385914118: ('0.10030687056887812',), 1385914115: ('0.10030687056887812',), 1385914109: ('0.10030687056887812',), 1385914108: ('0.10030687056887812',), 1385914105: ('0.10030687056887812',)} new output for save of step final : {1385917724: ('0.10030687056887812',), 1385917717: ('0.10030687056887812',), 1385917713: ('0.10030687056887812',), 1385917033: ('0.10030687056887812',), 1385917028: ('0.10030687056887812',), 1385916960: ('0.10030687056887812',), 1385916959: ('0.10030687056887812',), 1385914296: ('0.10030687056887812',), 1385914264: ('0.10030687056887812',), 1385914122: ('0.10030687056887812',), 1385914118: ('0.10030687056887812',), 1385914115: ('0.10030687056887812',), 1385914109: ('0.10030687056887812',), 1385914108: ('0.10030687056887812',), 1385914105: ('0.10030687056887812',)} [1385917724, 1385917717, 1385917713, 1385917033, 1385917028, 1385916960, 1385916959, 1385914296, 1385914264, 1385914122, 1385914118, 1385914115, 1385914109, 1385914108, 1385914105] Looping around the photos to save general results len do output : 15 /1385917724.Didn't retrieve data . /1385917717.Didn't retrieve data . /1385917713.Didn't retrieve data . /1385917033.Didn't retrieve data . /1385917028.Didn't retrieve data . /1385916960.Didn't retrieve data . /1385916959.Didn't retrieve data . /1385914296.Didn't retrieve data . /1385914264.Didn't retrieve data . /1385914122.Didn't retrieve data . /1385914118.Didn't retrieve data . /1385914115.Didn't retrieve data . /1385914109.Didn't retrieve data . /1385914108.Didn't retrieve data . /1385914105.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917724', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917717', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917713', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917033', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917028', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385916960', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385916959', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914296', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914264', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914122', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914118', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914115', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914109', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914108', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914105', None, None, None, None, None, '3772638') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 45 time used for this insertion : 0.013812065124511719 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.2590961456298828 time spend to save output : 0.01977396011352539 total time spend for step 11 : 0.2788701057434082 step12:velours_tree Fri Sep 26 11:51:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 1.4481985569000244 time spend to save output : 4.1484832763671875e-05 total time spend for step 12 : 1.448240041732788 step13:send_mail_cod Fri Sep 26 11:51:40 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin in order to get the selector url, please entre the license of selector results_Auto_P27193435_26-09-2025_11_51_40.pdf 27273407 imagette272734071758880300 27273409 imagette272734091758880300 27273411 imagette272734111758880300 27273413 imagette272734131758880300 27273415 change filename to text .imagette272734151758880300 27273419 imagette272734191758880300 27273422 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272734221758880300 27273414 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272734141758880302 27273416 imagette272734161758880304 27273418 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272734181758880304 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=27193435 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://marlene.fotonower.com/velours/27273428,27273429,27273430,27273431,27273432,27273433,27273434,27273435,27273436,27273437,27273438,27273439?tags=autre_refus,flou,papier,mal_croppe,metal,Teint_Dans_La_Masse,plastique,Carton_gris,Carton_brun,kraft,environnement,cartonnette your option no_mail is active, we will not send the real mail to your client args[1385917724] : ((1385917724, -0.08679068948437685, 496442774), (1385917724, 0.24040326405463058, 492688767), '0.10030687056887812') We are sending mail with results at report@fotonower.com args[1385917717] : ((1385917717, -0.08363955437369057, 496442774), (1385917717, -0.22623397330446895, 492688767), '0.10030687056887812') We are sending mail with results at report@fotonower.com args[1385917713] : ((1385917713, 0.11790323676924891, 2107752395), (1385917713, 0.1643390090455636, 492688767), '0.10030687056887812') We are sending mail with results at report@fotonower.com args[1385917033] : ((1385917033, 0.10781909497141791, 2107752395), (1385917033, 0.05502379523730346, 492688767), '0.10030687056887812') We are sending mail with results at report@fotonower.com args[1385917028] : ((1385917028, 0.00038497883336267586, 2107752395), (1385917028, 0.32017814490672764, 492688767), '0.10030687056887812') We are sending mail with results at report@fotonower.com args[1385916960] : ((1385916960, 0.09813655793697384, 2107752395), (1385916960, -2.5671850680881705, 492609224), '0.10030687056887812') We are sending mail with results at report@fotonower.com args[1385916959] : ((1385916959, -0.20080331680717733, 496442774), (1385916959, -0.09327647244606813, 492688767), '0.10030687056887812') We are sending mail with results at report@fotonower.com args[1385914296] : ((1385914296, 0.09813655793697384, 2107752395), (1385914296, -2.5671850680881705, 492609224), '0.10030687056887812') We are sending mail with results at report@fotonower.com args[1385914264] : ((1385914264, -0.20080331680717733, 496442774), (1385914264, -0.09327647244606813, 492688767), '0.10030687056887812') We are sending mail with results at report@fotonower.com args[1385914122] : ((1385914122, 0.03186425812392458, 2107752395), (1385914122, 0.5603198684518266, 492688767), '0.10030687056887812') We are sending mail with results at report@fotonower.com args[1385914118] : ((1385914118, 0.04474222902741933, 2107752395), (1385914118, 0.5345923916377042, 492688767), '0.10030687056887812') We are sending mail with results at report@fotonower.com args[1385914115] : ((1385914115, -0.07720461005015572, 496442774), (1385914115, 0.3362899798009341, 492688767), '0.10030687056887812') We are sending mail with results at report@fotonower.com args[1385914109] : ((1385914109, 0.07135618794875448, 2107752395), (1385914109, 0.29513063349902513, 492688767), '0.10030687056887812') We are sending mail with results at report@fotonower.com args[1385914108] : ((1385914108, 0.08560377413499641, 2107752395), (1385914108, 0.29371167790296515, 492688767), '0.10030687056887812') We are sending mail with results at report@fotonower.com args[1385914105] : ((1385914105, 0.07650710788485031, 2107752395), (1385914105, 0.441736436964093, 492688767), '0.10030687056887812') We are sending mail with results at report@fotonower.com refus_total : 0.10030687056887812 2022-04-13 10:29:59 0 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27193435_26-09-2025_11_51_40.pdf results_Auto_P27193435_26-09-2025_11_51_40.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27193435_26-09-2025_11_51_40.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4433','27193435','results_Auto_P27193435_26-09-2025_11_51_40.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27193435_26-09-2025_11_51_40.pdf','pdf','','1.33','0.10030687056887812') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1385917724, 1385917717, 1385917713, 1385917033, 1385917028, 1385916960, 1385916959, 1385914296, 1385914264, 1385914122, 1385914118, 1385914115, 1385914109, 1385914108, 1385914105] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917724', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917717', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917713', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917033', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917028', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385916960', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385916959', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914296', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914264', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914122', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914118', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914115', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914109', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914108', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914105', None, None, None, None, None, '3772638') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 15 time used for this insertion : 0.015161275863647461 save_final save missing photos in datou_result : time spend for datou_step_exec : 11.54317331314087 time spend to save output : 0.019253015518188477 total time spend for step 13 : 11.562426328659058 step14:split_time_score Fri Sep 26 11:51:52 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('10', 30),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 24092025 27193435 Nombre de photos uploadées : 30 / 23040 (0%) 24092025 27193435 Nombre de photos taguées (types de déchets): 0 / 30 (0%) 24092025 27193435 Nombre de photos taguées (volume) : 0 / 30 (0%) elapsed_time : load_data_split_time_score 2.384185791015625e-06 elapsed_time : order_list_meta_photo_and_scores 6.67572021484375e-06 ?????????????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0029528141021728516 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.22178339958190918 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.09096285938141155 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27193431_26-09-2025_11_45_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27193431 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27193431 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27193432 order by id desc limit 1 Qualite : 0.0994719176427741 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27193433_26-09-2025_11_51_38.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27193433 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27193433 AND mptpi.`type`=3726 To do Qualite : 0.11091927628898955 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27193435_26-09-2025_11_51_40.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27193435 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27193435 AND mptpi.`type`=3726 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'24092025': {'nb_upload': 30, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1385917724, 1385917717, 1385917713, 1385917033, 1385917028, 1385916960, 1385916959, 1385914296, 1385914264, 1385914122, 1385914118, 1385914115, 1385914109, 1385914108, 1385914105] Looping around the photos to save general results len do output : 1 /27193435Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917724', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917717', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917713', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917033', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385917028', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385916960', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385916959', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914296', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914264', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914122', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914118', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914115', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914109', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914108', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', None, '1385914105', None, None, None, None, None, '3772638') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 16 time used for this insertion : 0.014871358871459961 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.5678584575653076 time spend to save output : 0.01509404182434082 total time spend for step 14 : 1.5829524993896484 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 333.46user 311.72system 12:00.08elapsed 89%CPU (0avgtext+0avgdata 5469128maxresident)k 8605896inputs+369752outputs (258568major+29570147minor)pagefaults 0swaps