python /home/admin/mtr/script_for_cron.py -j default -m 20 -a 'python3 ~/workarea/git/Velours/python/prod/datou.py -j batch_current -a 4433' -s traitement_sts -M 0 -S 0 -U 100,80,95 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/home/admin/workarea/git/apy', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 66326 load datou : 4433 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4433, datou_cur_ids : ['3772638'] with mtr_portfolio_ids : ['27193435'] and first list_photo_ids : [] new path : /proc/66326/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, brightness, blur_detection, crop_condition, thcl, tfhub_classification2, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 15 list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 15 ; length of list_pids : 15 ; length of list_args : 15 time to download the photos : 4.689587593078613 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 14 step1:mask_detect Fri Sep 26 11:27:33 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 6800 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-09-26 11:27:36.428708: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-26 11:27:36.456583: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-26 11:27:36.458793: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fbe8c000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-26 11:27:36.458860: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-26 11:27:36.463272: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-26 11:27:36.606021: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1f1e88b0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-26 11:27:36.606076: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-26 11:27:36.607286: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:27:36.607712: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:27:36.611008: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:27:36.614042: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:27:36.614479: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:27:36.617267: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:27:36.618721: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:27:36.624497: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:27:36.625956: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:27:36.626043: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:27:36.626686: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-26 11:27:36.626703: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-26 11:27:36.626713: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-26 11:27:36.627827: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6105 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-09-26 11:27:36.915941: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:27:36.916058: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:27:36.916091: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:27:36.916129: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:27:36.916161: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:27:36.916190: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:27:36.916218: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:27:36.916251: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:27:36.917740: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:27:36.918882: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:27:36.918918: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:27:36.918934: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:27:36.918948: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:27:36.918962: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:27:36.918976: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:27:36.918990: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:27:36.919004: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:27:36.920050: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:27:36.920096: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-26 11:27:36.920106: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-26 11:27:36.920114: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-26 11:27:36.921216: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6105 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-09-26 11:27:53.039766: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:27:53.443025: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 15 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 3.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 13 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 15 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 5.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 11 Detection mask done ! Trying to reset tf kernel 66580 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 9 tf kernel not reseted sub process len(results) : 15 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 15 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 6800 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.001611471176147461 nb_pixel_total : 41947 time to create 1 rle with old method : 0.04673600196838379 length of segment : 349 time for calcul the mask position with numpy : 0.0024242401123046875 nb_pixel_total : 110342 time to create 1 rle with old method : 0.11822175979614258 length of segment : 335 time for calcul the mask position with numpy : 0.0006070137023925781 nb_pixel_total : 28615 time to create 1 rle with old method : 0.031317949295043945 length of segment : 283 time for calcul the mask position with numpy : 0.24950504302978516 nb_pixel_total : 890854 time to create 1 rle with new method : 0.5214278697967529 length of segment : 2414 time for calcul the mask position with numpy : 0.003956794738769531 nb_pixel_total : 123263 time to create 1 rle with old method : 0.13006329536437988 length of segment : 418 time for calcul the mask position with numpy : 0.004319190979003906 nb_pixel_total : 239245 time to create 1 rle with new method : 0.013872623443603516 length of segment : 600 time for calcul the mask position with numpy : 0.002228975296020508 nb_pixel_total : 128191 time to create 1 rle with old method : 0.13632655143737793 length of segment : 464 time for calcul the mask position with numpy : 0.0017457008361816406 nb_pixel_total : 97983 time to create 1 rle with old method : 0.10261344909667969 length of segment : 781 time for calcul the mask position with numpy : 0.0009195804595947266 nb_pixel_total : 46142 time to create 1 rle with old method : 0.06709647178649902 length of segment : 344 time for calcul the mask position with numpy : 0.002309560775756836 nb_pixel_total : 126505 time to create 1 rle with old method : 0.15201687812805176 length of segment : 643 time for calcul the mask position with numpy : 0.0009138584136962891 nb_pixel_total : 47067 time to create 1 rle with old method : 0.054715871810913086 length of segment : 275 time for calcul the mask position with numpy : 0.0014369487762451172 nb_pixel_total : 84080 time to create 1 rle with old method : 0.09430289268493652 length of segment : 328 time for calcul the mask position with numpy : 0.0005388259887695312 nb_pixel_total : 24745 time to create 1 rle with old method : 0.028532743453979492 length of segment : 292 time for calcul the mask position with numpy : 0.027178049087524414 nb_pixel_total : 708330 time to create 1 rle with new method : 0.25249409675598145 length of segment : 2335 time for calcul the mask position with numpy : 0.0003561973571777344 nb_pixel_total : 18027 time to create 1 rle with old method : 0.021152257919311523 length of segment : 206 time for calcul the mask position with numpy : 0.0012044906616210938 nb_pixel_total : 69806 time to create 1 rle with old method : 0.07990384101867676 length of segment : 444 time for calcul the mask position with numpy : 0.00020432472229003906 nb_pixel_total : 6537 time to create 1 rle with old method : 0.019084453582763672 length of segment : 86 time for calcul the mask position with numpy : 0.0023446083068847656 nb_pixel_total : 138994 time to create 1 rle with old method : 0.15491294860839844 length of segment : 454 time for calcul the mask position with numpy : 0.0003609657287597656 nb_pixel_total : 17748 time to create 1 rle with old method : 0.020097732543945312 length of segment : 137 time for calcul the mask position with numpy : 0.009426355361938477 nb_pixel_total : 588093 time to create 1 rle with new method : 0.1256542205810547 length of segment : 855 time for calcul the mask position with numpy : 0.00040841102600097656 nb_pixel_total : 23693 time to create 1 rle with old method : 0.031076669692993164 length of segment : 233 time for calcul the mask position with numpy : 0.001538991928100586 nb_pixel_total : 116489 time to create 1 rle with old method : 0.1316208839416504 length of segment : 387 time for calcul the mask position with numpy : 0.0004875659942626953 nb_pixel_total : 13735 time to create 1 rle with old method : 0.01584148406982422 length of segment : 148 time for calcul the mask position with numpy : 0.0005052089691162109 nb_pixel_total : 15532 time to create 1 rle with old method : 0.018164873123168945 length of segment : 185 time for calcul the mask position with numpy : 0.005708932876586914 nb_pixel_total : 174469 time to create 1 rle with new method : 0.010699987411499023 length of segment : 762 time for calcul the mask position with numpy : 0.005378246307373047 nb_pixel_total : 282312 time to create 1 rle with new method : 0.00924062728881836 length of segment : 810 time for calcul the mask position with numpy : 0.002160310745239258 nb_pixel_total : 86067 time to create 1 rle with old method : 0.09851717948913574 length of segment : 218 time for calcul the mask position with numpy : 0.0015461444854736328 nb_pixel_total : 52823 time to create 1 rle with old method : 0.060436248779296875 length of segment : 293 time for calcul the mask position with numpy : 0.0036351680755615234 nb_pixel_total : 161459 time to create 1 rle with new method : 0.005330562591552734 length of segment : 402 time for calcul the mask position with numpy : 0.0005633831024169922 nb_pixel_total : 18819 time to create 1 rle with old method : 0.021633625030517578 length of segment : 139 time for calcul the mask position with numpy : 0.006214141845703125 nb_pixel_total : 210248 time to create 1 rle with new method : 0.012716293334960938 length of segment : 1022 time for calcul the mask position with numpy : 0.0019812583923339844 nb_pixel_total : 97921 time to create 1 rle with old method : 0.11202430725097656 length of segment : 471 time for calcul the mask position with numpy : 0.0004875659942626953 nb_pixel_total : 18010 time to create 1 rle with old method : 0.020911455154418945 length of segment : 320 time for calcul the mask position with numpy : 0.04378628730773926 nb_pixel_total : 1422495 time to create 1 rle with new method : 0.8392388820648193 length of segment : 2760 time for calcul the mask position with numpy : 0.0002770423889160156 nb_pixel_total : 6675 time to create 1 rle with old method : 0.008090734481811523 length of segment : 71 time for calcul the mask position with numpy : 0.00495457649230957 nb_pixel_total : 272013 time to create 1 rle with new method : 0.008147954940795898 length of segment : 648 time for calcul the mask position with numpy : 0.006147861480712891 nb_pixel_total : 332654 time to create 1 rle with new method : 0.01091313362121582 length of segment : 560 time for calcul the mask position with numpy : 0.0020453929901123047 nb_pixel_total : 68608 time to create 1 rle with old method : 0.07884383201599121 length of segment : 499 time for calcul the mask position with numpy : 0.004996538162231445 nb_pixel_total : 344268 time to create 1 rle with new method : 0.012625932693481445 length of segment : 580 time for calcul the mask position with numpy : 0.000370025634765625 nb_pixel_total : 12963 time to create 1 rle with old method : 0.015297174453735352 length of segment : 112 time for calcul the mask position with numpy : 0.0027747154235839844 nb_pixel_total : 117414 time to create 1 rle with old method : 0.15862250328063965 length of segment : 419 time for calcul the mask position with numpy : 0.0047533512115478516 nb_pixel_total : 265170 time to create 1 rle with new method : 0.008313894271850586 length of segment : 537 time for calcul the mask position with numpy : 0.0070002079010009766 nb_pixel_total : 307941 time to create 1 rle with new method : 0.011690378189086914 length of segment : 728 time for calcul the mask position with numpy : 0.00048160552978515625 nb_pixel_total : 12707 time to create 1 rle with old method : 0.01859116554260254 length of segment : 72 time for calcul the mask position with numpy : 0.0014603137969970703 nb_pixel_total : 64921 time to create 1 rle with old method : 0.07480740547180176 length of segment : 307 time for calcul the mask position with numpy : 0.00406336784362793 nb_pixel_total : 138960 time to create 1 rle with old method : 0.1577892303466797 length of segment : 392 time for calcul the mask position with numpy : 0.001985311508178711 nb_pixel_total : 79569 time to create 1 rle with old method : 0.08977985382080078 length of segment : 243 time for calcul the mask position with numpy : 0.0013380050659179688 nb_pixel_total : 46060 time to create 1 rle with old method : 0.05202007293701172 length of segment : 248 time for calcul the mask position with numpy : 0.003358125686645508 nb_pixel_total : 170341 time to create 1 rle with new method : 0.01032400131225586 length of segment : 502 time for calcul the mask position with numpy : 0.007883548736572266 nb_pixel_total : 360030 time to create 1 rle with new method : 0.0166323184967041 length of segment : 858 time for calcul the mask position with numpy : 0.0010695457458496094 nb_pixel_total : 82138 time to create 1 rle with old method : 0.09116411209106445 length of segment : 349 time for calcul the mask position with numpy : 0.031232357025146484 nb_pixel_total : 419352 time to create 1 rle with new method : 0.4312169551849365 length of segment : 637 time for calcul the mask position with numpy : 0.004309654235839844 nb_pixel_total : 213799 time to create 1 rle with new method : 0.00822758674621582 length of segment : 601 time for calcul the mask position with numpy : 0.002054929733276367 nb_pixel_total : 89850 time to create 1 rle with old method : 0.10036134719848633 length of segment : 352 time for calcul the mask position with numpy : 0.0013723373413085938 nb_pixel_total : 67548 time to create 1 rle with old method : 0.07655811309814453 length of segment : 265 time for calcul the mask position with numpy : 0.05162525177001953 nb_pixel_total : 1378837 time to create 1 rle with new method : 0.6753597259521484 length of segment : 2289 time for calcul the mask position with numpy : 0.009789228439331055 nb_pixel_total : 596616 time to create 1 rle with new method : 0.04082989692687988 length of segment : 1228 time for calcul the mask position with numpy : 0.00829315185546875 nb_pixel_total : 542678 time to create 1 rle with new method : 0.09945201873779297 length of segment : 1070 time for calcul the mask position with numpy : 0.015171289443969727 nb_pixel_total : 729107 time to create 1 rle with new method : 0.12123918533325195 length of segment : 1238 time for calcul the mask position with numpy : 0.0017452239990234375 nb_pixel_total : 63741 time to create 1 rle with old method : 0.06975531578063965 length of segment : 400 time for calcul the mask position with numpy : 0.0009560585021972656 nb_pixel_total : 30105 time to create 1 rle with old method : 0.03400921821594238 length of segment : 204 time for calcul the mask position with numpy : 0.00033736228942871094 nb_pixel_total : 10633 time to create 1 rle with old method : 0.012832880020141602 length of segment : 85 time for calcul the mask position with numpy : 0.08346700668334961 nb_pixel_total : 1533055 time to create 1 rle with new method : 0.29260683059692383 length of segment : 2555 time for calcul the mask position with numpy : 0.001428842544555664 nb_pixel_total : 50235 time to create 1 rle with old method : 0.056688547134399414 length of segment : 321 time for calcul the mask position with numpy : 0.007384538650512695 nb_pixel_total : 191759 time to create 1 rle with new method : 0.015197992324829102 length of segment : 486 time for calcul the mask position with numpy : 0.0070133209228515625 nb_pixel_total : 390360 time to create 1 rle with new method : 0.012482166290283203 length of segment : 675 time for calcul the mask position with numpy : 0.01049351692199707 nb_pixel_total : 470395 time to create 1 rle with new method : 0.02671194076538086 length of segment : 1034 time for calcul the mask position with numpy : 0.002470731735229492 nb_pixel_total : 198538 time to create 1 rle with new method : 0.006498098373413086 length of segment : 467 time for calcul the mask position with numpy : 0.0007789134979248047 nb_pixel_total : 34698 time to create 1 rle with old method : 0.057326555252075195 length of segment : 206 time for calcul the mask position with numpy : 0.0035181045532226562 nb_pixel_total : 199079 time to create 1 rle with new method : 0.007009029388427734 length of segment : 517 time for calcul the mask position with numpy : 0.003292560577392578 nb_pixel_total : 199384 time to create 1 rle with new method : 0.00879359245300293 length of segment : 576 time for calcul the mask position with numpy : 0.0009205341339111328 nb_pixel_total : 22717 time to create 1 rle with old method : 0.024747848510742188 length of segment : 229 time for calcul the mask position with numpy : 0.0026493072509765625 nb_pixel_total : 135740 time to create 1 rle with old method : 0.14851951599121094 length of segment : 505 time for calcul the mask position with numpy : 0.007584571838378906 nb_pixel_total : 353652 time to create 1 rle with new method : 0.015175342559814453 length of segment : 779 time for calcul the mask position with numpy : 0.0001590251922607422 nb_pixel_total : 1666 time to create 1 rle with old method : 0.001976490020751953 length of segment : 69 time for calcul the mask position with numpy : 0.003975629806518555 nb_pixel_total : 147269 time to create 1 rle with old method : 0.1625831127166748 length of segment : 517 time for calcul the mask position with numpy : 0.0017786026000976562 nb_pixel_total : 79253 time to create 1 rle with old method : 0.08985447883605957 length of segment : 413 time for calcul the mask position with numpy : 0.038901329040527344 nb_pixel_total : 1639210 time to create 1 rle with new method : 0.549992561340332 length of segment : 2667 time for calcul the mask position with numpy : 0.0034618377685546875 nb_pixel_total : 195618 time to create 1 rle with new method : 0.007002353668212891 length of segment : 895 time for calcul the mask position with numpy : 0.0011093616485595703 nb_pixel_total : 69824 time to create 1 rle with old method : 0.07630062103271484 length of segment : 389 time for calcul the mask position with numpy : 0.0017054080963134766 nb_pixel_total : 38036 time to create 1 rle with old method : 0.04354572296142578 length of segment : 492 time for calcul the mask position with numpy : 0.0011703968048095703 nb_pixel_total : 40521 time to create 1 rle with old method : 0.045671939849853516 length of segment : 303 time for calcul the mask position with numpy : 0.03664731979370117 nb_pixel_total : 1602247 time to create 1 rle with new method : 0.5560207366943359 length of segment : 2707 time for calcul the mask position with numpy : 0.001300811767578125 nb_pixel_total : 46072 time to create 1 rle with old method : 0.05362534523010254 length of segment : 217 time for calcul the mask position with numpy : 0.000820159912109375 nb_pixel_total : 46461 time to create 1 rle with old method : 0.05181145668029785 length of segment : 232 time for calcul the mask position with numpy : 0.0020415782928466797 nb_pixel_total : 52780 time to create 1 rle with old method : 0.05580449104309082 length of segment : 473 time for calcul the mask position with numpy : 0.0014731884002685547 nb_pixel_total : 95929 time to create 1 rle with old method : 0.10689306259155273 length of segment : 575 time for calcul the mask position with numpy : 0.00032067298889160156 nb_pixel_total : 7900 time to create 1 rle with old method : 0.009198665618896484 length of segment : 113 time for calcul the mask position with numpy : 0.015340089797973633 nb_pixel_total : 595265 time to create 1 rle with new method : 0.030042171478271484 length of segment : 1019 time for calcul the mask position with numpy : 0.0040738582611083984 nb_pixel_total : 241187 time to create 1 rle with new method : 0.00775146484375 length of segment : 506 time for calcul the mask position with numpy : 0.050427913665771484 nb_pixel_total : 2312585 time to create 1 rle with new method : 0.5804116725921631 length of segment : 3098 time for calcul the mask position with numpy : 0.0015234947204589844 nb_pixel_total : 86249 time to create 1 rle with old method : 0.09803271293640137 length of segment : 294 time for calcul the mask position with numpy : 0.004236459732055664 nb_pixel_total : 289424 time to create 1 rle with new method : 0.01143503189086914 length of segment : 834 time for calcul the mask position with numpy : 0.0014874935150146484 nb_pixel_total : 86553 time to create 1 rle with old method : 0.09555816650390625 length of segment : 302 time for calcul the mask position with numpy : 0.009110689163208008 nb_pixel_total : 490739 time to create 1 rle with new method : 0.01713252067565918 length of segment : 1107 time for calcul the mask position with numpy : 0.0006885528564453125 nb_pixel_total : 21565 time to create 1 rle with old method : 0.024100303649902344 length of segment : 245 time for calcul the mask position with numpy : 0.048488616943359375 nb_pixel_total : 2863753 time to create 1 rle with new method : 0.31771063804626465 length of segment : 2525 time for calcul the mask position with numpy : 0.0008542537689208984 nb_pixel_total : 40875 time to create 1 rle with old method : 0.045487403869628906 length of segment : 139 time for calcul the mask position with numpy : 0.0076007843017578125 nb_pixel_total : 296985 time to create 1 rle with new method : 0.01455235481262207 length of segment : 855 time for calcul the mask position with numpy : 0.0037429332733154297 nb_pixel_total : 182833 time to create 1 rle with new method : 0.00575566291809082 length of segment : 411 time for calcul the mask position with numpy : 0.00020599365234375 nb_pixel_total : 2659 time to create 1 rle with old method : 0.0033156871795654297 length of segment : 52 time for calcul the mask position with numpy : 0.0021605491638183594 nb_pixel_total : 46671 time to create 1 rle with old method : 0.05528140068054199 length of segment : 322 time for calcul the mask position with numpy : 0.0006878376007080078 nb_pixel_total : 39667 time to create 1 rle with old method : 0.050914764404296875 length of segment : 146 time for calcul the mask position with numpy : 0.024586915969848633 nb_pixel_total : 681651 time to create 1 rle with new method : 0.17407512664794922 length of segment : 2186 time for calcul the mask position with numpy : 0.002825021743774414 nb_pixel_total : 122870 time to create 1 rle with old method : 0.13396334648132324 length of segment : 373 time for calcul the mask position with numpy : 0.27925729751586914 nb_pixel_total : 2395007 time to create 1 rle with new method : 1.0115993022918701 length of segment : 4276 time for calcul the mask position with numpy : 0.0004677772521972656 nb_pixel_total : 30895 time to create 1 rle with old method : 0.034714698791503906 length of segment : 193 time for calcul the mask position with numpy : 0.0048062801361083984 nb_pixel_total : 274154 time to create 1 rle with new method : 0.008132457733154297 length of segment : 496 time for calcul the mask position with numpy : 0.00028228759765625 nb_pixel_total : 12118 time to create 1 rle with old method : 0.019551515579223633 length of segment : 181 time for calcul the mask position with numpy : 0.006762981414794922 nb_pixel_total : 345700 time to create 1 rle with new method : 0.012161016464233398 length of segment : 692 time for calcul the mask position with numpy : 0.00244140625 nb_pixel_total : 186523 time to create 1 rle with new method : 0.005968332290649414 length of segment : 561 time for calcul the mask position with numpy : 0.000164031982421875 nb_pixel_total : 3907 time to create 1 rle with old method : 0.004883766174316406 length of segment : 61 time for calcul the mask position with numpy : 0.015411376953125 nb_pixel_total : 967212 time to create 1 rle with new method : 0.0836484432220459 length of segment : 1533 time for calcul the mask position with numpy : 0.0013327598571777344 nb_pixel_total : 53623 time to create 1 rle with old method : 0.05883502960205078 length of segment : 270 time for calcul the mask position with numpy : 0.003554821014404297 nb_pixel_total : 182026 time to create 1 rle with new method : 0.006279706954956055 length of segment : 362 time for calcul the mask position with numpy : 0.0022385120391845703 nb_pixel_total : 194159 time to create 1 rle with new method : 0.00468754768371582 length of segment : 341 time for calcul the mask position with numpy : 0.0011758804321289062 nb_pixel_total : 82987 time to create 1 rle with old method : 0.09241437911987305 length of segment : 358 time for calcul the mask position with numpy : 0.004080772399902344 nb_pixel_total : 183653 time to create 1 rle with new method : 0.007647991180419922 length of segment : 538 time for calcul the mask position with numpy : 0.0005018711090087891 nb_pixel_total : 10891 time to create 1 rle with old method : 0.01201772689819336 length of segment : 170 time for calcul the mask position with numpy : 0.0023071765899658203 nb_pixel_total : 98688 time to create 1 rle with old method : 0.11026430130004883 length of segment : 415 time for calcul the mask position with numpy : 0.001367807388305664 nb_pixel_total : 56854 time to create 1 rle with old method : 0.06573343276977539 length of segment : 355 time for calcul the mask position with numpy : 0.0014011859893798828 nb_pixel_total : 62427 time to create 1 rle with old method : 0.07724809646606445 length of segment : 370 time for calcul the mask position with numpy : 0.039148807525634766 nb_pixel_total : 1304691 time to create 1 rle with new method : 0.21766877174377441 length of segment : 2395 time for calcul the mask position with numpy : 0.010444164276123047 nb_pixel_total : 555696 time to create 1 rle with new method : 0.12372970581054688 length of segment : 1574 time for calcul the mask position with numpy : 0.002694368362426758 nb_pixel_total : 216314 time to create 1 rle with new method : 0.006303071975708008 length of segment : 427 time for calcul the mask position with numpy : 0.00510096549987793 nb_pixel_total : 291911 time to create 1 rle with new method : 0.01488184928894043 length of segment : 550 time for calcul the mask position with numpy : 0.0004661083221435547 nb_pixel_total : 9087 time to create 1 rle with old method : 0.010448455810546875 length of segment : 174 time spent for convertir_results : 35.17645478248596 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 127 chid ids of type : 3663 Number RLEs to save : 84805 save missing photos in datou_result : time spend for datou_step_exec : 140.1896870136261 time spend to save output : 4.570276737213135 total time spend for step 1 : 144.75996375083923 step2:brightness Fri Sep 26 11:29:58 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1758878848_66326_1385918666_b3fb4f0798a903e67ef7853de2875ce6.jpg treat image : temp/1758878848_66326_1385918663_3ca653efb859ed7c46ef70e1d8b1a225.jpg treat image : temp/1758878848_66326_1385918662_3d027f9f00e82d10faeb2503f7277b01.jpg treat image : temp/1758878848_66326_1385918658_5c0e97bbd9991d904ee1c19e538e4f90.jpg treat image : temp/1758878848_66326_1385918656_a0717c8fcae815cc4031cc93c78efaac.jpg treat image : temp/1758878848_66326_1385918654_e294e1499a29b636fad175ffb7ef858e.jpg treat image : temp/1758878848_66326_1385918381_7d8789dca642cd1ad24ee3c70e8df947.jpg treat image : temp/1758878848_66326_1385918376_053e2851d8f0a276d855560a47517cd6.jpg treat image : temp/1758878848_66326_1385918361_969b57e0a72fa93d12fb2e5649dd054b.jpg treat image : temp/1758878848_66326_1385918335_d836e1315d4f50f63d3535b257f58918.jpg treat image : temp/1758878848_66326_1385918323_00e9b551f93757cfc8112b6f2382b095.jpg treat image : temp/1758878848_66326_1385918320_dd977ce8469432e8ae309f362f7cc5a6.jpg treat image : temp/1758878848_66326_1385917740_1619491f88ad284a9d8646b69a8b8ffe.jpg treat image : temp/1758878848_66326_1385917732_f2fc953803c7d7595196f97d43fb0e69.jpg treat image : temp/1758878848_66326_1385917729_4ed375a6dda4857416e239432ac307d4.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 15 time used for this insertion : 0.01544499397277832 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 15 time used for this insertion : 0.019218921661376953 save missing photos in datou_result : time spend for datou_step_exec : 24.386890411376953 time spend to save output : 0.039278507232666016 total time spend for step 2 : 24.42616891860962 step3:blur_detection Fri Sep 26 11:30:22 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1758878848_66326_1385918666_b3fb4f0798a903e67ef7853de2875ce6.jpg resize: (3040, 4056) 1385918666 0.19278024592505497 treat image : temp/1758878848_66326_1385918663_3ca653efb859ed7c46ef70e1d8b1a225.jpg resize: (3040, 4056) 1385918663 0.11011084547759217 treat image : temp/1758878848_66326_1385918662_3d027f9f00e82d10faeb2503f7277b01.jpg resize: (3040, 4056) 1385918662 0.3851614687982524 treat image : temp/1758878848_66326_1385918658_5c0e97bbd9991d904ee1c19e538e4f90.jpg resize: (3040, 4056) 1385918658 0.4648148929826399 treat image : temp/1758878848_66326_1385918656_a0717c8fcae815cc4031cc93c78efaac.jpg resize: (3040, 4056) 1385918656 0.4366129658881093 treat image : temp/1758878848_66326_1385918654_e294e1499a29b636fad175ffb7ef858e.jpg resize: (3040, 4056) 1385918654 -0.038099871318829415 treat image : temp/1758878848_66326_1385918381_7d8789dca642cd1ad24ee3c70e8df947.jpg resize: (3040, 4056) 1385918381 -0.08043742140382161 treat image : temp/1758878848_66326_1385918376_053e2851d8f0a276d855560a47517cd6.jpg resize: (3040, 4056) 1385918376 -0.20958555245810098 treat image : temp/1758878848_66326_1385918361_969b57e0a72fa93d12fb2e5649dd054b.jpg resize: (3040, 4056) 1385918361 0.1427283774103872 treat image : temp/1758878848_66326_1385918335_d836e1315d4f50f63d3535b257f58918.jpg resize: (3040, 4056) 1385918335 -0.2700593440680864 treat image : temp/1758878848_66326_1385918323_00e9b551f93757cfc8112b6f2382b095.jpg resize: (3040, 4056) 1385918323 0.40758924724434176 treat image : temp/1758878848_66326_1385918320_dd977ce8469432e8ae309f362f7cc5a6.jpg resize: (3040, 4056) 1385918320 0.5596086512127206 treat image : temp/1758878848_66326_1385917740_1619491f88ad284a9d8646b69a8b8ffe.jpg resize: (3040, 4056) 1385917740 -0.262324382504418 treat image : temp/1758878848_66326_1385917732_f2fc953803c7d7595196f97d43fb0e69.jpg resize: (3040, 4056) 1385917732 0.2753493946975951 treat image : temp/1758878848_66326_1385917729_4ed375a6dda4857416e239432ac307d4.jpg resize: (3040, 4056) 1385917729 0.024410769871462765 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 15 time used for this insertion : 0.013902664184570312 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 15 time used for this insertion : 0.0128326416015625 save missing photos in datou_result : time spend for datou_step_exec : 92.35478949546814 time spend to save output : 0.4949803352355957 total time spend for step 3 : 92.84976983070374 step4:crop_condition Fri Sep 26 11:31:55 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 15 ! batch 1 Loaded 127 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 17 About to insert : list_path_to_insert length 17 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 11 About to insert : list_path_to_insert length 11 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 22 About to insert : list_path_to_insert length 22 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 23 About to insert : list_path_to_insert length 23 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1385918666, 1385918663, 1385918662, 1385918658, 1385918656, 1385918654, 1385918381, 1385918376, 1385918361, 1385918335, 1385918323, 1385918320, 1385917740, 1385917732, 1385917729] Looping around the photos to save general results len do output : 85 /-3976647506Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647522Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647524Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647526Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647527Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647528Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647541Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647547Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647558Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647571Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647577Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647578Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647586Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647588Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647593Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647594Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647625Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647529Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647555Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647561Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647603Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647609Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647628Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647630Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647511Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647512Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647535Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647542Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647543Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647553Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647574Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647580Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647581Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647607Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647619Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647509Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647513Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647516Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647520Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647533Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647537Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647545Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647546Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647550Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647551Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647562Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647570Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647572Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647575Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647597Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647602Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647604Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647612Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647614Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647615Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647620Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647626Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647505Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647507Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647514Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647515Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647521Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647523Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647525Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647531Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647532Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647540Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647549Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647552Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647559Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647564Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647566Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647568Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647589Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647596Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647600Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647611Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647613Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647618Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647624Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647557Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647563Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647591Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647599Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976647617Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918666', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918663', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918662', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918658', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918656', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918654', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918381', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918376', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918361', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918335', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918323', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918320', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917740', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917732', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917729', None, None, None, None, None, '3772638') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 270 time used for this insertion : 0.042815446853637695 save_final save missing photos in datou_result : time spend for datou_step_exec : 99.07397127151489 time spend to save output : 0.1382598876953125 total time spend for step 4 : 99.2122311592102 step5:thcl Fri Sep 26 11:33:34 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 1 we are using the classfication for multi_thcl [2456] time to import caffe and check if the image exist : 0.017017364501953125 time to convert the images to numpy array : 0.42637085914611816 time to import caffe and check if the image exist : 0.019215822219848633 time to convert the images to numpy array : 0.44220495223999023 time to import caffe and check if the image exist : 0.020946979522705078 time to convert the images to numpy array : 0.4438965320587158 time to import caffe and check if the image exist : 0.012839317321777344 time to convert the images to numpy array : 0.6273653507232666 time to import caffe and check if the image exist : 0.014746427536010742 time to convert the images to numpy array : 0.6661713123321533 time to import caffe and check if the image exist : 0.010274648666381836 time to convert the images to numpy array : 0.7401745319366455 time to import caffe and check if the image exist : 0.0180971622467041 time to convert the images to numpy array : 1.1468639373779297 time to import caffe and check if the image exist : 0.020051002502441406 time to convert the images to numpy array : 1.1484568119049072 time to import caffe and check if the image exist : 0.015195846557617188 time to convert the images to numpy array : 1.1619558334350586 time to import caffe and check if the image exist : 0.016649723052978516 time to convert the images to numpy array : 1.1820518970489502 total time to convert the images to numpy array : 1.5221350193023682 list photo_ids error: [] list photo_ids correct : [-3976647568, -3976647589, -3976647596, -3976647600, -3976647611, -3976647613, -3976647618, -3976647624, -3976647557, -3976647525, -3976647531, -3976647532, -3976647540, -3976647549, -3976647552, -3976647559, -3976647564, -3976647566, -3976647615, -3976647620, -3976647626, -3976647505, -3976647507, -3976647514, -3976647515, -3976647521, -3976647523, -3976647513, -3976647516, -3976647520, -3976647533, -3976647537, -3976647545, -3976647546, -3976647550, -3976647551, -3976647542, -3976647543, -3976647553, -3976647574, -3976647580, -3976647581, -3976647607, -3976647619, -3976647509, -3976647506, -3976647522, -3976647524, -3976647526, -3976647527, -3976647528, -3976647541, -3976647547, -3976647558, -3976647571, -3976647577, -3976647578, -3976647586, -3976647588, -3976647593, -3976647594, -3976647625, -3976647529, -3976647562, -3976647570, -3976647572, -3976647575, -3976647597, -3976647602, -3976647604, -3976647612, -3976647614, -3976647555, -3976647561, -3976647603, -3976647609, -3976647628, -3976647630, -3976647511, -3976647512, -3976647535, -3976647563, -3976647591, -3976647599, -3976647617] number of photos to traite : 85 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10356 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/:/home/admin/workarea/git/apy/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10137 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 2.1267905235290527 time used to do the prediction : 0.19678306579589844 we don't save the descriptors for this thcl 2456 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 9.298324584960938e-06 save missing photos in datou_result : time spend for datou_step_exec : 9.023741006851196 time spend to save output : 0.00030541419982910156 total time spend for step 5 : 9.024046421051025 step6:tfhub_classification2 Fri Sep 26 11:33:43 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou_step TFHub with tf2 ! we are using the classfication for only one thcl 3501 begin to check gpu status inside check gpu memory 2025-09-26 11:33:46.696292: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-26 11:33:46.697108: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:33:46.697192: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:33:46.697236: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:33:46.699294: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:33:46.699351: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:33:46.701470: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:33:46.702448: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:33:46.706479: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:33:46.707747: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:33:46.708242: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-26 11:33:46.732614: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-26 11:33:46.734518: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fbd78000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-26 11:33:46.734567: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-26 11:33:46.738271: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2a120350 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-26 11:33:46.738302: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-26 11:33:46.739599: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:33:46.739702: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:33:46.739735: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:33:46.739865: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:33:46.739900: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:33:46.739946: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:33:46.739996: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:33:46.740047: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:33:46.741790: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:33:46.741863: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:33:46.741924: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-26 11:33:46.741940: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-26 11:33:46.741953: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-26 11:33:46.743748: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3096 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) l 3637 free memory gpu now : 10135 max_wait_temp : 1 max_wait : 5 1 Physical GPUs, 1 Logical GPUs tagging for thcl : 3501 To do loadFromThcl(), then load ParamDescType : thcl3501 thcls : [{'id': 3501, 'mtr_user_id': 31, 'name': 'learn_papier_mixte_centre_tf_b4_220223_3', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '9087724,9087727,9087725,9087729,9087726,9087723,9087722,9087720,9087721,9087728', 'photo_hashtag_type': 4539, 'photo_desc_type': 5759, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 3501, 'mtr_user_id': 31, 'name': 'learn_papier_mixte_centre_tf_b4_220223_3', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '9087724,9087727,9087725,9087729,9087726,9087723,9087722,9087720,9087721,9087728', 'photo_hashtag_type': 4539, 'photo_desc_type': 5759, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5759 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5759, 'learn_papier_mixte_centre_tf_b4_220223_3', 1792, 1792, 'learn_papier_mixte_centre_tf_b4_220223_3', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 2, 22, 13, 54, 32), datetime.datetime(2023, 2, 22, 13, 54, 32)) model_name : learn_papier_mixte_centre_tf_b4_220223_3 model_param file didn't exist model_name : learn_papier_mixte_centre_tf_b4_220223_3 model_type : tf_classification2 list file need : ['Confusion_Matrix.png', 'Precision_Recall_Carton_brun.jpg', 'Precision_Recall_Carton_gris.jpg', 'Precision_Recall_Teint_Dans_La_Masse.jpg', 'Precision_Recall_autre_refus.jpg', 'Precision_Recall_cartonnette.jpg', 'Precision_Recall_environnement.jpg', 'Precision_Recall_kraft.jpg', 'Precision_Recall_metal.jpg', 'Precision_Recall_papier.jpg', 'Precision_Recall_plastique.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file exist in s3 : ['Confusion_Matrix.png', 'Precision_Recall_Carton_brun.jpg', 'Precision_Recall_Carton_gris.jpg', 'Precision_Recall_Teint_Dans_La_Masse.jpg', 'Precision_Recall_autre_refus.jpg', 'Precision_Recall_cartonnette.jpg', 'Precision_Recall_environnement.jpg', 'Precision_Recall_kraft.jpg', 'Precision_Recall_metal.jpg', 'Precision_Recall_papier.jpg', 'Precision_Recall_plastique.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file manque in s3 : [] local folder : /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3 /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Confusion_Matrix.png size_local : 121269 size in s3 : 121269 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:17 Confusion_Matrix.png already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_Carton_brun.jpg size_local : 76104 size in s3 : 76104 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_Carton_brun.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_Carton_gris.jpg size_local : 82912 size in s3 : 82912 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_Carton_gris.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_Teint_Dans_La_Masse.jpg size_local : 75423 size in s3 : 75423 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:25 Precision_Recall_Teint_Dans_La_Masse.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_autre_refus.jpg size_local : 78883 size in s3 : 78883 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_autre_refus.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_cartonnette.jpg size_local : 81791 size in s3 : 81791 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:30 Precision_Recall_cartonnette.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_environnement.jpg size_local : 59936 size in s3 : 59936 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:17 Precision_Recall_environnement.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_kraft.jpg size_local : 80280 size in s3 : 80280 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:25 Precision_Recall_kraft.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_metal.jpg size_local : 77736 size in s3 : 77736 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_metal.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_papier.jpg size_local : 69041 size in s3 : 69041 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_papier.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_plastique.jpg size_local : 70770 size in s3 : 70770 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:17 Precision_Recall_plastique.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Result_Summary.txt size_local : 1092 size in s3 : 1092 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Result_Summary.txt already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/checkpoint size_local : 99 size in s3 : 99 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 checkpoint already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_checkpoint.ckpt.data-00000-of-00002 size_local : 429801 size in s3 : 429801 create time local : 2023-03-03 12:26:16 create time in s3 : 2023-02-22 12:54:30 model_checkpoint.ckpt.data-00000-of-00002 already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_checkpoint.ckpt.data-00001-of-00002 size_local : 141033188 size in s3 : 141033188 create time local : 2023-03-03 12:26:17 create time in s3 : 2023-02-22 12:54:25 model_checkpoint.ckpt.data-00001-of-00002 already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_checkpoint.ckpt.index size_local : 85618 size in s3 : 85618 create time local : 2023-03-03 12:26:17 create time in s3 : 2023-02-22 12:54:30 model_checkpoint.ckpt.index already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_weights.h5 size_local : 71289448 size in s3 : 71289448 create time local : 2023-03-03 12:26:18 create time in s3 : 2023-02-22 12:54:17 model_weights.h5 already exist and didn't need to update desc size : 1792 Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= module (KerasLayer) (None, 1792) 17673816 _________________________________________________________________ learn_papier_mixte_centre_tf (None, 10) 17930 ================================================================= Total params: 17,691,746 Trainable params: 17,930 Non-trainable params: 17,673,816 _________________________________________________________________ Loading Weights... time used to create the model : 17.08839774131775 time used to load_weights : 0.4199700355529785 0it [00:00, ?it/s] 75it [00:00, 742.66it/s] 85it [00:00, 730.52it/s]2025-09-26 11:34:07.499234: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:34:07.521187: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 temp/1758878848_66326_1385918666_b3fb4f0798a903e67ef7853de2875ce6_rle_crop_3976647506_0.png temp/1758878848_66326_1385918662_3d027f9f00e82d10faeb2503f7277b01_rle_crop_3976647522_0.png temp/1758878848_66326_1385918662_3d027f9f00e82d10faeb2503f7277b01_rle_crop_3976647524_0.png temp/1758878848_66326_1385918662_3d027f9f00e82d10faeb2503f7277b01_rle_crop_3976647526_0.png temp/1758878848_66326_1385918662_3d027f9f00e82d10faeb2503f7277b01_rle_crop_3976647527_0.png temp/1758878848_66326_1385918662_3d027f9f00e82d10faeb2503f7277b01_rle_crop_3976647528_0.png temp/1758878848_66326_1385918656_a0717c8fcae815cc4031cc93c78efaac_rle_crop_3976647541_0.png temp/1758878848_66326_1385918654_e294e1499a29b636fad175ffb7ef858e_rle_crop_3976647547_0.png temp/1758878848_66326_1385918381_7d8789dca642cd1ad24ee3c70e8df947_rle_crop_3976647558_0.png temp/1758878848_66326_1385918376_053e2851d8f0a276d855560a47517cd6_rle_crop_3976647571_0.png temp/1758878848_66326_1385918361_969b57e0a72fa93d12fb2e5649dd054b_rle_crop_3976647577_0.png temp/1758878848_66326_1385918361_969b57e0a72fa93d12fb2e5649dd054b_rle_crop_3976647578_0.png temp/1758878848_66326_1385918335_d836e1315d4f50f63d3535b257f58918_rle_crop_3976647586_0.png temp/1758878848_66326_1385918335_d836e1315d4f50f63d3535b257f58918_rle_crop_3976647588_0.png temp/1758878848_66326_1385918323_00e9b551f93757cfc8112b6f2382b095_rle_crop_3976647593_0.png temp/1758878848_66326_1385918323_00e9b551f93757cfc8112b6f2382b095_rle_crop_3976647594_0.png temp/1758878848_66326_1385917729_4ed375a6dda4857416e239432ac307d4_rle_crop_3976647625_0.png temp/1758878848_66326_1385918662_3d027f9f00e82d10faeb2503f7277b01_rle_crop_3976647529_0.png temp/1758878848_66326_1385918654_e294e1499a29b636fad175ffb7ef858e_rle_crop_3976647555_0.png temp/1758878848_66326_1385918381_7d8789dca642cd1ad24ee3c70e8df947_rle_crop_3976647561_0.png temp/1758878848_66326_1385918320_dd977ce8469432e8ae309f362f7cc5a6_rle_crop_3976647603_0.png temp/1758878848_66326_1385918320_dd977ce8469432e8ae309f362f7cc5a6_rle_crop_3976647609_0.png temp/1758878848_66326_1385917729_4ed375a6dda4857416e239432ac307d4_rle_crop_3976647628_0.png temp/1758878848_66326_1385917729_4ed375a6dda4857416e239432ac307d4_rle_crop_3976647630_0.png temp/1758878848_66326_1385918666_b3fb4f0798a903e67ef7853de2875ce6_rle_crop_3976647511_0.png temp/1758878848_66326_1385918666_b3fb4f0798a903e67ef7853de2875ce6_rle_crop_3976647512_0.png temp/1758878848_66326_1385918658_5c0e97bbd9991d904ee1c19e538e4f90_rle_crop_3976647535_0.png temp/1758878848_66326_1385918656_a0717c8fcae815cc4031cc93c78efaac_rle_crop_3976647542_0.png temp/1758878848_66326_1385918656_a0717c8fcae815cc4031cc93c78efaac_rle_crop_3976647543_0.png temp/1758878848_66326_1385918654_e294e1499a29b636fad175ffb7ef858e_rle_crop_3976647553_0.png temp/1758878848_66326_1385918376_053e2851d8f0a276d855560a47517cd6_rle_crop_3976647574_0.png temp/1758878848_66326_1385918361_969b57e0a72fa93d12fb2e5649dd054b_rle_crop_3976647580_0.png temp/1758878848_66326_1385918361_969b57e0a72fa93d12fb2e5649dd054b_rle_crop_3976647581_0.png temp/1758878848_66326_1385918320_dd977ce8469432e8ae309f362f7cc5a6_rle_crop_3976647607_0.png temp/1758878848_66326_1385917732_f2fc953803c7d7595196f97d43fb0e69_rle_crop_3976647619_0.png temp/1758878848_66326_1385918666_b3fb4f0798a903e67ef7853de2875ce6_rle_crop_3976647509_0.png temp/1758878848_66326_1385918666_b3fb4f0798a903e67ef7853de2875ce6_rle_crop_3976647513_0.png temp/1758878848_66326_1385918663_3ca653efb859ed7c46ef70e1d8b1a225_rle_crop_3976647516_0.png temp/1758878848_66326_1385918663_3ca653efb859ed7c46ef70e1d8b1a225_rle_crop_3976647520_0.png temp/1758878848_66326_1385918658_5c0e97bbd9991d904ee1c19e538e4f90_rle_crop_3976647533_0.png temp/1758878848_66326_1385918658_5c0e97bbd9991d904ee1c19e538e4f90_rle_crop_3976647537_0.png temp/1758878848_66326_1385918656_a0717c8fcae815cc4031cc93c78efaac_rle_crop_3976647545_0.png temp/1758878848_66326_1385918656_a0717c8fcae815cc4031cc93c78efaac_rle_crop_3976647546_0.png temp/1758878848_66326_1385918654_e294e1499a29b636fad175ffb7ef858e_rle_crop_3976647550_0.png temp/1758878848_66326_1385918654_e294e1499a29b636fad175ffb7ef858e_rle_crop_3976647551_0.png temp/1758878848_66326_1385918381_7d8789dca642cd1ad24ee3c70e8df947_rle_crop_3976647562_0.png temp/1758878848_66326_1385918376_053e2851d8f0a276d855560a47517cd6_rle_crop_3976647570_0.png temp/1758878848_66326_1385918376_053e2851d8f0a276d855560a47517cd6_rle_crop_3976647572_0.png temp/1758878848_66326_1385918376_053e2851d8f0a276d855560a47517cd6_rle_crop_3976647575_0.png temp/1758878848_66326_1385918323_00e9b551f93757cfc8112b6f2382b095_rle_crop_3976647597_0.png temp/1758878848_66326_1385918320_dd977ce8469432e8ae309f362f7cc5a6_rle_crop_3976647602_0.png temp/1758878848_66326_1385918320_dd977ce8469432e8ae309f362f7cc5a6_rle_crop_3976647604_0.png temp/1758878848_66326_1385917740_1619491f88ad284a9d8646b69a8b8ffe_rle_crop_3976647612_0.png temp/1758878848_66326_1385917740_1619491f88ad284a9d8646b69a8b8ffe_rle_crop_3976647614_0.png temp/1758878848_66326_1385917740_1619491f88ad284a9d8646b69a8b8ffe_rle_crop_3976647615_0.png temp/1758878848_66326_1385917732_f2fc953803c7d7595196f97d43fb0e69_rle_crop_3976647620_0.png temp/1758878848_66326_1385917729_4ed375a6dda4857416e239432ac307d4_rle_crop_3976647626_0.png temp/1758878848_66326_1385918666_b3fb4f0798a903e67ef7853de2875ce6_rle_crop_3976647505_0.png temp/1758878848_66326_1385918666_b3fb4f0798a903e67ef7853de2875ce6_rle_crop_3976647507_0.png temp/1758878848_66326_1385918666_b3fb4f0798a903e67ef7853de2875ce6_rle_crop_3976647514_0.png temp/1758878848_66326_1385918663_3ca653efb859ed7c46ef70e1d8b1a225_rle_crop_3976647515_0.png temp/1758878848_66326_1385918662_3d027f9f00e82d10faeb2503f7277b01_rle_crop_3976647521_0.png temp/1758878848_66326_1385918662_3d027f9f00e82d10faeb2503f7277b01_rle_crop_3976647523_0.png temp/1758878848_66326_1385918662_3d027f9f00e82d10faeb2503f7277b01_rle_crop_3976647525_0.png temp/1758878848_66326_1385918658_5c0e97bbd9991d904ee1c19e538e4f90_rle_crop_3976647531_0.png temp/1758878848_66326_1385918658_5c0e97bbd9991d904ee1c19e538e4f90_rle_crop_3976647532_0.png temp/1758878848_66326_1385918656_a0717c8fcae815cc4031cc93c78efaac_rle_crop_3976647540_0.png temp/1758878848_66326_1385918654_e294e1499a29b636fad175ffb7ef858e_rle_crop_3976647549_0.png temp/1758878848_66326_1385918654_e294e1499a29b636fad175ffb7ef858e_rle_crop_3976647552_0.png temp/1758878848_66326_1385918381_7d8789dca642cd1ad24ee3c70e8df947_rle_crop_3976647559_0.png temp/1758878848_66326_1385918376_053e2851d8f0a276d855560a47517cd6_rle_crop_3976647564_0.png temp/1758878848_66326_1385918376_053e2851d8f0a276d855560a47517cd6_rle_crop_3976647566_0.png temp/1758878848_66326_1385918376_053e2851d8f0a276d855560a47517cd6_rle_crop_3976647568_0.png temp/1758878848_66326_1385918335_d836e1315d4f50f63d3535b257f58918_rle_crop_3976647589_0.png temp/1758878848_66326_1385918323_00e9b551f93757cfc8112b6f2382b095_rle_crop_3976647596_0.png temp/1758878848_66326_1385918320_dd977ce8469432e8ae309f362f7cc5a6_rle_crop_3976647600_0.png temp/1758878848_66326_1385917740_1619491f88ad284a9d8646b69a8b8ffe_rle_crop_3976647611_0.png temp/1758878848_66326_1385917740_1619491f88ad284a9d8646b69a8b8ffe_rle_crop_3976647613_0.png temp/1758878848_66326_1385917732_f2fc953803c7d7595196f97d43fb0e69_rle_crop_3976647618_0.png temp/1758878848_66326_1385917729_4ed375a6dda4857416e239432ac307d4_rle_crop_3976647624_0.png temp/1758878848_66326_1385918654_e294e1499a29b636fad175ffb7ef858e_rle_crop_3976647557_0.png temp/1758878848_66326_1385918381_7d8789dca642cd1ad24ee3c70e8df947_rle_crop_3976647563_0.png temp/1758878848_66326_1385918335_d836e1315d4f50f63d3535b257f58918_rle_crop_3976647591_0.png temp/1758878848_66326_1385918323_00e9b551f93757cfc8112b6f2382b095_rle_crop_3976647599_0.png temp/1758878848_66326_1385917740_1619491f88ad284a9d8646b69a8b8ffe_rle_crop_3976647617_0.png Found 85 images belonging to 1 classes. begin to do the prediction : time used to do the prediction : 6.854829788208008 (85,) (85, 10) (85, 1792) shape of features : (85, 1792) shape of new features : (1, 85, 1792) save descriptor for thcl : 3501 time to traite the descriptors : 0.47457146644592285 storage_type for insertDescriptorsMulti : 3 Missing photo l117 : -3976647506 Missing photo l117 : -3976647522 Missing photo l117 : -3976647524 Missing photo l117 : -3976647526 Missing photo l117 : -3976647527 Missing photo l117 : -3976647528 Missing photo l117 : -3976647541 Missing photo l117 : -3976647547 Missing photo l117 : -3976647558 Missing photo l117 : -3976647571 Missing photo l117 : -3976647577 Missing photo l117 : -3976647578 Missing photo l117 : -3976647586 Missing photo l117 : -3976647588 Missing photo l117 : -3976647593 Missing photo l117 : -3976647594 Missing photo l117 : -3976647625 Missing photo l117 : -3976647529 Missing photo l117 : -3976647555 Missing photo l117 : -3976647561 Missing photo l117 : -3976647603 Missing photo l117 : -3976647609 Missing photo l117 : -3976647628 Missing photo l117 : -3976647630 Missing photo l117 : -3976647511 Missing photo l117 : -3976647512 Missing photo l117 : -3976647535 Missing photo l117 : -3976647542 Missing photo l117 : -3976647543 Missing photo l117 : -3976647553 Missing photo l117 : -3976647574 Missing photo l117 : -3976647580 Missing photo l117 : -3976647581 Missing photo l117 : -3976647607 Missing photo l117 : -3976647619 Missing photo l117 : -3976647509 Missing photo l117 : -3976647513 Missing photo l117 : -3976647516 Missing photo l117 : -3976647520 Missing photo l117 : -3976647533 Missing photo l117 : -3976647537 Missing photo l117 : -3976647545 Missing photo l117 : -3976647546 Missing photo l117 : -3976647550 Missing photo l117 : -3976647551 Missing photo l117 : -3976647562 Missing photo l117 : -3976647570 Missing photo l117 : -3976647572 Missing photo l117 : -3976647575 Missing photo l117 : -3976647597 Missing photo l117 : -3976647602 Missing photo l117 : -3976647604 Missing photo l117 : -3976647612 Missing photo l117 : -3976647614 Missing photo l117 : -3976647615 Missing photo l117 : -3976647620 Missing photo l117 : -3976647626 Missing photo l117 : -3976647505 Missing photo l117 : -3976647507 Missing photo l117 : -3976647514 Missing photo l117 : -3976647515 Missing photo l117 : -3976647521 Missing photo l117 : -3976647523 Missing photo l117 : -3976647525 Missing photo l117 : -3976647531 Missing photo l117 : -3976647532 Missing photo l117 : -3976647540 Missing photo l117 : -3976647549 Missing photo l117 : -3976647552 Missing photo l117 : -3976647559 Missing photo l117 : -3976647564 Missing photo l117 : -3976647566 Missing photo l117 : -3976647568 Missing photo l117 : -3976647589 Missing photo l117 : -3976647596 Missing photo l117 : -3976647600 Missing photo l117 : -3976647611 Missing photo l117 : -3976647613 Missing photo l117 : -3976647618 Missing photo l117 : -3976647624 Missing photo l117 : -3976647557 Missing photo l117 : -3976647563 Missing photo l117 : -3976647591 Missing photo l117 : -3976647599 Missing photo l117 : -3976647617 To insert : -3976647506 Missing photo l134 : -3976647506 To insert : -3976647522 Missing photo l134 : -3976647522 To insert : -3976647524 Missing photo l134 : -3976647524 To insert : -3976647526 Missing photo l134 : -3976647526 To insert : -3976647527 Missing photo l134 : -3976647527 To insert : -3976647528 Missing photo l134 : -3976647528 To insert : -3976647541 Missing photo l134 : -3976647541 To insert : -3976647547 Missing photo l134 : -3976647547 To insert : -3976647558 Missing photo l134 : -3976647558 To insert : -3976647571 Missing photo l134 : -3976647571 To insert : -3976647577 Missing photo l134 : -3976647577 To insert : -3976647578 Missing photo l134 : -3976647578 To insert : -3976647586 Missing photo l134 : -3976647586 To insert : -3976647588 Missing photo l134 : -3976647588 To insert : -3976647593 Missing photo l134 : -3976647593 To insert : -3976647594 Missing photo l134 : -3976647594 To insert : -3976647625 Missing photo l134 : -3976647625 To insert : -3976647529 Missing photo l134 : -3976647529 To insert : -3976647555 Missing photo l134 : -3976647555 To insert : -3976647561 Missing photo l134 : -3976647561 To insert : -3976647603 Missing photo l134 : -3976647603 To insert : -3976647609 Missing photo l134 : -3976647609 To insert : -3976647628 Missing photo l134 : -3976647628 To insert : -3976647630 Missing photo l134 : -3976647630 To insert : -3976647511 Missing photo l134 : -3976647511 To insert : -3976647512 Missing photo l134 : -3976647512 To insert : -3976647535 Missing photo l134 : -3976647535 To insert : -3976647542 Missing photo l134 : -3976647542 To insert : -3976647543 Missing photo l134 : -3976647543 To insert : -3976647553 Missing photo l134 : -3976647553 To insert : -3976647574 Missing photo l134 : -3976647574 To insert : -3976647580 Missing photo l134 : -3976647580 To insert : -3976647581 Missing photo l134 : -3976647581 To insert : -3976647607 Missing photo l134 : -3976647607 To insert : -3976647619 Missing photo l134 : -3976647619 To insert : -3976647509 Missing photo l134 : -3976647509 To insert : -3976647513 Missing photo l134 : -3976647513 To insert : -3976647516 Missing photo l134 : -3976647516 To insert : -3976647520 Missing photo l134 : -3976647520 To insert : -3976647533 Missing photo l134 : -3976647533 To insert : -3976647537 Missing photo l134 : -3976647537 To insert : -3976647545 Missing photo l134 : -3976647545 To insert : -3976647546 Missing photo l134 : -3976647546 To insert : -3976647550 Missing photo l134 : -3976647550 To insert : -3976647551 Missing photo l134 : -3976647551 To insert : -3976647562 Missing photo l134 : -3976647562 To insert : -3976647570 Missing photo l134 : -3976647570 To insert : -3976647572 Missing photo l134 : -3976647572 To insert : -3976647575 Missing photo l134 : -3976647575 To insert : -3976647597 Missing photo l134 : -3976647597 To insert : -3976647602 Missing photo l134 : -3976647602 To insert : -3976647604 Missing photo l134 : -3976647604 To insert : -3976647612 Missing photo l134 : -3976647612 To insert : -3976647614 Missing photo l134 : -3976647614 To insert : -3976647615 Missing photo l134 : -3976647615 To insert : -3976647620 Missing photo l134 : -3976647620 To insert : -3976647626 Missing photo l134 : -3976647626 To insert : -3976647505 Missing photo l134 : -3976647505 To insert : -3976647507 Missing photo l134 : -3976647507 To insert : -3976647514 Missing photo l134 : -3976647514 To insert : -3976647515 Missing photo l134 : -3976647515 To insert : -3976647521 Missing photo l134 : -3976647521 To insert : -3976647523 Missing photo l134 : -3976647523 To insert : -3976647525 Missing photo l134 : -3976647525 To insert : -3976647531 Missing photo l134 : -3976647531 To insert : -3976647532 Missing photo l134 : -3976647532 To insert : -3976647540 Missing photo l134 : -3976647540 To insert : -3976647549 Missing photo l134 : -3976647549 To insert : -3976647552 Missing photo l134 : -3976647552 To insert : -3976647559 Missing photo l134 : -3976647559 To insert : -3976647564 Missing photo l134 : -3976647564 To insert : -3976647566 Missing photo l134 : -3976647566 To insert : -3976647568 Missing photo l134 : -3976647568 To insert : -3976647589 Missing photo l134 : -3976647589 To insert : -3976647596 Missing photo l134 : -3976647596 To insert : -3976647600 Missing photo l134 : -3976647600 To insert : -3976647611 Missing photo l134 : -3976647611 To insert : -3976647613 Missing photo l134 : -3976647613 To insert : -3976647618 Missing photo l134 : -3976647618 To insert : -3976647624 Missing photo l134 : -3976647624 To insert : -3976647557 Missing photo l134 : -3976647557 To insert : -3976647563 Missing photo l134 : -3976647563 To insert : -3976647591 Missing photo l134 : -3976647591 To insert : -3976647599 Missing photo l134 : -3976647599 To insert : -3976647617 Missing photo l134 : -3976647617 time to insert the descriptors : 17.050293922424316 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : tfhub_classification2 we use saveGeneral [1385918666, 1385918663, 1385918662, 1385918658, 1385918656, 1385918654, 1385918381, 1385918376, 1385918361, 1385918335, 1385918323, 1385918320, 1385917740, 1385917732, 1385917729] Looping around the photos to save general results len do output : 85 /-3976647506Didn't retrieve data . /-3976647522Didn't retrieve data . /-3976647524Didn't retrieve data . /-3976647526Didn't retrieve data . /-3976647527Didn't retrieve data . /-3976647528Didn't retrieve data . /-3976647541Didn't retrieve data . /-3976647547Didn't retrieve data . /-3976647558Didn't retrieve data . /-3976647571Didn't retrieve data . /-3976647577Didn't retrieve data . /-3976647578Didn't retrieve data . /-3976647586Didn't retrieve data . /-3976647588Didn't retrieve data . /-3976647593Didn't retrieve data . /-3976647594Didn't retrieve data . /-3976647625Didn't retrieve data . /-3976647529Didn't retrieve data . /-3976647555Didn't retrieve data . /-3976647561Didn't retrieve data . /-3976647603Didn't retrieve data . /-3976647609Didn't retrieve data . /-3976647628Didn't retrieve data . /-3976647630Didn't retrieve data . /-3976647511Didn't retrieve data . /-3976647512Didn't retrieve data . /-3976647535Didn't retrieve data . /-3976647542Didn't retrieve data . /-3976647543Didn't retrieve data . /-3976647553Didn't retrieve data . /-3976647574Didn't retrieve data . /-3976647580Didn't retrieve data . /-3976647581Didn't retrieve data . /-3976647607Didn't retrieve data . /-3976647619Didn't retrieve data . /-3976647509Didn't retrieve data . /-3976647513Didn't retrieve data . /-3976647516Didn't retrieve data . /-3976647520Didn't retrieve data . /-3976647533Didn't retrieve data . /-3976647537Didn't retrieve data . /-3976647545Didn't retrieve data . /-3976647546Didn't retrieve data . /-3976647550Didn't retrieve data . /-3976647551Didn't retrieve data . /-3976647562Didn't retrieve data . /-3976647570Didn't retrieve data . /-3976647572Didn't retrieve data . /-3976647575Didn't retrieve data . /-3976647597Didn't retrieve data . /-3976647602Didn't retrieve data . /-3976647604Didn't retrieve data . /-3976647612Didn't retrieve data . /-3976647614Didn't retrieve data . /-3976647615Didn't retrieve data . /-3976647620Didn't retrieve data . /-3976647626Didn't retrieve data . /-3976647505Didn't retrieve data . /-3976647507Didn't retrieve data . /-3976647514Didn't retrieve data . /-3976647515Didn't retrieve data . /-3976647521Didn't retrieve data . /-3976647523Didn't retrieve data . /-3976647525Didn't retrieve data . /-3976647531Didn't retrieve data . /-3976647532Didn't retrieve data . /-3976647540Didn't retrieve data . /-3976647549Didn't retrieve data . /-3976647552Didn't retrieve data . /-3976647559Didn't retrieve data . /-3976647564Didn't retrieve data . /-3976647566Didn't retrieve data . /-3976647568Didn't retrieve data . /-3976647589Didn't retrieve data . /-3976647596Didn't retrieve data . /-3976647600Didn't retrieve data . /-3976647611Didn't retrieve data . /-3976647613Didn't retrieve data . /-3976647618Didn't retrieve data . /-3976647624Didn't retrieve data . /-3976647557Didn't retrieve data . /-3976647563Didn't retrieve data . /-3976647591Didn't retrieve data . /-3976647599Didn't retrieve data . /-3976647617Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918666', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918663', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918662', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918658', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918656', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918654', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918381', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918376', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918361', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918335', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918323', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918320', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917740', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917732', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917729', None, None, None, None, None, '3772638') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 100 time used for this insertion : 0.029400110244750977 save_final save missing photos in datou_result : time spend for datou_step_exec : 48.143362283706665 time spend to save output : 0.036200761795043945 total time spend for step 6 : 48.17956304550171 step7:merge_mask_thcl_custom Fri Sep 26 11:34:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 127 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1385918666, 1385918663, 1385918662, 1385918658, 1385918656, 1385918654, 1385918381, 1385918376, 1385918361, 1385918335, 1385918323, 1385918320, 1385917740, 1385917732, 1385917729] Looping around the photos to save general results len do output : 15 /1385918666Didn't retrieve data .Didn't retrieve data . /1385918663Didn't retrieve data .Didn't retrieve data . /1385918662Didn't retrieve data .Didn't retrieve data . /1385918658Didn't retrieve data .Didn't retrieve data . /1385918656Didn't retrieve data .Didn't retrieve data . /1385918654Didn't retrieve data .Didn't retrieve data . /1385918381Didn't retrieve data .Didn't retrieve data . /1385918376Didn't retrieve data .Didn't retrieve data . /1385918361Didn't retrieve data .Didn't retrieve data . /1385918335Didn't retrieve data .Didn't retrieve data . /1385918323Didn't retrieve data .Didn't retrieve data . /1385918320Didn't retrieve data .Didn't retrieve data . /1385917740Didn't retrieve data .Didn't retrieve data . /1385917732Didn't retrieve data .Didn't retrieve data . /1385917729Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918666', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918663', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918662', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918658', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918656', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918654', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918381', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918376', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918361', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918335', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918323', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918320', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917740', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917732', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917729', None, None, None, None, None, '3772638') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 45 time used for this insertion : 0.017782926559448242 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.06656765937805176 time spend to save output : 0.018544435501098633 total time spend for step 7 : 0.08511209487915039 step8:rle_unique_nms_with_priority Fri Sep 26 11:34:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 10 nb_hashtags : 4 time to prepare the origin masks : 11.558839559555054 time for calcul the mask position with numpy : 0.5055274963378906 nb_pixel_total : 10712080 time to create 1 rle with new method : 0.968548059463501 time for calcul the mask position with numpy : 0.06178927421569824 nb_pixel_total : 4764 time to create 1 rle with old method : 0.005992412567138672 time for calcul the mask position with numpy : 0.0717930793762207 nb_pixel_total : 41958 time to create 1 rle with old method : 0.04659628868103027 time for calcul the mask position with numpy : 0.06481266021728516 nb_pixel_total : 125620 time to create 1 rle with old method : 0.1495513916015625 time for calcul the mask position with numpy : 0.0694122314453125 nb_pixel_total : 46096 time to create 1 rle with old method : 0.059890031814575195 time for calcul the mask position with numpy : 0.07651114463806152 nb_pixel_total : 28596 time to create 1 rle with old method : 0.03634786605834961 time for calcul the mask position with numpy : 0.07376360893249512 nb_pixel_total : 238971 time to create 1 rle with new method : 1.8700191974639893 time for calcul the mask position with numpy : 0.04648613929748535 nb_pixel_total : 5878 time to create 1 rle with old method : 0.007648468017578125 time for calcul the mask position with numpy : 0.04292035102844238 nb_pixel_total : 123278 time to create 1 rle with old method : 0.14423203468322754 time for calcul the mask position with numpy : 0.0563201904296875 nb_pixel_total : 893101 time to create 1 rle with new method : 0.9741809368133545 time for calcul the mask position with numpy : 0.04413151741027832 nb_pixel_total : 109898 time to create 1 rle with old method : 0.13107776641845703 create new chi : 5.667267322540283 time to delete rle : 0.016722917556762695 batch 1 Loaded 11 chid ids of type : 3726 Number RLEs to save : 14570 TO DO : save crop sub photo not yet done ! save time : 0.8516111373901367 nb_obj : 5 nb_hashtags : 4 time to prepare the origin masks : 8.691143989562988 time for calcul the mask position with numpy : 0.7843880653381348 nb_pixel_total : 11403113 time to create 1 rle with new method : 1.1051292419433594 time for calcul the mask position with numpy : 0.04529094696044922 nb_pixel_total : 64848 time to create 1 rle with old method : 0.07385516166687012 time for calcul the mask position with numpy : 0.03928375244140625 nb_pixel_total : 83794 time to create 1 rle with old method : 0.09423446655273438 time for calcul the mask position with numpy : 0.04365348815917969 nb_pixel_total : 706989 time to create 1 rle with new method : 1.1848373413085938 time for calcul the mask position with numpy : 0.0381314754486084 nb_pixel_total : 24632 time to create 1 rle with old method : 0.02776646614074707 time for calcul the mask position with numpy : 0.03955817222595215 nb_pixel_total : 46864 time to create 1 rle with old method : 0.05240678787231445 create new chi : 3.648550271987915 time to delete rle : 0.0011866092681884766 batch 1 Loaded 6 chid ids of type : 3726 Number RLEs to save : 10143 TO DO : save crop sub photo not yet done ! save time : 0.6801302433013916 nb_obj : 9 nb_hashtags : 5 time to prepare the origin masks : 11.799709558486938 time for calcul the mask position with numpy : 0.6345686912536621 nb_pixel_total : 11235535 time to create 1 rle with new method : 0.88710618019104 time for calcul the mask position with numpy : 0.05456852912902832 nb_pixel_total : 588514 time to create 1 rle with new method : 0.9207212924957275 time for calcul the mask position with numpy : 0.059578895568847656 nb_pixel_total : 174275 time to create 1 rle with new method : 0.7651588916778564 time for calcul the mask position with numpy : 0.05887126922607422 nb_pixel_total : 15489 time to create 1 rle with old method : 0.017812728881835938 time for calcul the mask position with numpy : 0.05827045440673828 nb_pixel_total : 17665 time to create 1 rle with old method : 0.020925045013427734 time for calcul the mask position with numpy : 0.04387521743774414 nb_pixel_total : 23672 time to create 1 rle with old method : 0.027758359909057617 time for calcul the mask position with numpy : 0.04742884635925293 nb_pixel_total : 116206 time to create 1 rle with old method : 0.13082098960876465 time for calcul the mask position with numpy : 0.04629349708557129 nb_pixel_total : 138662 time to create 1 rle with old method : 0.15903043746948242 time for calcul the mask position with numpy : 0.04756641387939453 nb_pixel_total : 13701 time to create 1 rle with old method : 0.020435571670532227 time for calcul the mask position with numpy : 0.04337358474731445 nb_pixel_total : 6521 time to create 1 rle with old method : 0.0075800418853759766 create new chi : 4.2246315479278564 time to delete rle : 0.0010449886322021484 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 9505 TO DO : save crop sub photo not yet done ! save time : 0.834496259689331 nb_obj : 7 nb_hashtags : 4 time to prepare the origin masks : 12.25684142112732 time for calcul the mask position with numpy : 0.5589542388916016 nb_pixel_total : 10352937 time to create 1 rle with new method : 1.243328332901001 time for calcul the mask position with numpy : 0.06816911697387695 nb_pixel_total : 156886 time to create 1 rle with new method : 0.8657708168029785 time for calcul the mask position with numpy : 0.07673120498657227 nb_pixel_total : 1422093 time to create 1 rle with new method : 1.0081672668457031 time for calcul the mask position with numpy : 0.06968212127685547 nb_pixel_total : 17943 time to create 1 rle with old method : 0.03574991226196289 time for calcul the mask position with numpy : 0.07600855827331543 nb_pixel_total : 223129 time to create 1 rle with new method : 1.100407361984253 time for calcul the mask position with numpy : 0.07189750671386719 nb_pixel_total : 52771 time to create 1 rle with old method : 0.06714701652526855 time for calcul the mask position with numpy : 0.058899641036987305 nb_pixel_total : 18792 time to create 1 rle with old method : 0.02077507972717285 time for calcul the mask position with numpy : 0.059311866760253906 nb_pixel_total : 85689 time to create 1 rle with old method : 0.10019063949584961 create new chi : 5.695603370666504 time to delete rle : 0.002294778823852539 batch 1 Loaded 8 chid ids of type : 3726 Number RLEs to save : 12767 TO DO : save crop sub photo not yet done ! save time : 0.8100576400756836 nb_obj : 8 nb_hashtags : 3 time to prepare the origin masks : 10.20651626586914 time for calcul the mask position with numpy : 0.6227869987487793 nb_pixel_total : 11242805 time to create 1 rle with new method : 0.9652485847473145 time for calcul the mask position with numpy : 0.06993222236633301 nb_pixel_total : 2390 time to create 1 rle with old method : 0.003384828567504883 time for calcul the mask position with numpy : 0.05942201614379883 nb_pixel_total : 343693 time to create 1 rle with new method : 0.7656400203704834 time for calcul the mask position with numpy : 0.06682777404785156 nb_pixel_total : 271840 time to create 1 rle with new method : 2.474045753479004 time for calcul the mask position with numpy : 0.06222367286682129 nb_pixel_total : 68307 time to create 1 rle with old method : 0.07652807235717773 time for calcul the mask position with numpy : 0.06530594825744629 nb_pixel_total : 264507 time to create 1 rle with new method : 0.7591242790222168 time for calcul the mask position with numpy : 0.06179165840148926 nb_pixel_total : 117134 time to create 1 rle with old method : 0.13891386985778809 time for calcul the mask position with numpy : 0.06185650825500488 nb_pixel_total : 12913 time to create 1 rle with old method : 0.01844644546508789 time for calcul the mask position with numpy : 0.061025381088256836 nb_pixel_total : 6651 time to create 1 rle with old method : 0.00962972640991211 create new chi : 6.5367631912231445 time to delete rle : 0.0017032623291015625 batch 1 Loaded 9 chid ids of type : 3726 Number RLEs to save : 9078 TO DO : save crop sub photo not yet done ! save time : 0.5528335571289062 nb_obj : 10 nb_hashtags : 6 time to prepare the origin masks : 12.38940715789795 time for calcul the mask position with numpy : 0.625741720199585 nb_pixel_total : 11043220 time to create 1 rle with new method : 1.585627794265747 time for calcul the mask position with numpy : 0.07156014442443848 nb_pixel_total : 32626 time to create 1 rle with old method : 0.04422163963317871 time for calcul the mask position with numpy : 0.046649932861328125 nb_pixel_total : 18223 time to create 1 rle with old method : 0.025031566619873047 time for calcul the mask position with numpy : 0.05225348472595215 nb_pixel_total : 79361 time to create 1 rle with old method : 0.0940701961517334 time for calcul the mask position with numpy : 0.04290032386779785 nb_pixel_total : 138648 time to create 1 rle with old method : 0.1757824420928955 time for calcul the mask position with numpy : 0.043068647384643555 nb_pixel_total : 373205 time to create 1 rle with new method : 0.8111605644226074 time for calcul the mask position with numpy : 0.0712883472442627 nb_pixel_total : 64439 time to create 1 rle with old method : 0.07423615455627441 time for calcul the mask position with numpy : 0.06864309310913086 nb_pixel_total : 45911 time to create 1 rle with old method : 0.050374746322631836 time for calcul the mask position with numpy : 0.07290077209472656 nb_pixel_total : 213990 time to create 1 rle with new method : 0.6659536361694336 time for calcul the mask position with numpy : 0.07808589935302734 nb_pixel_total : 307984 time to create 1 rle with new method : 0.7589883804321289 time for calcul the mask position with numpy : 0.07651042938232422 nb_pixel_total : 12633 time to create 1 rle with old method : 0.013664960861206055 create new chi : 5.760342121124268 time to delete rle : 0.002069711685180664 batch 1 Loaded 11 chid ids of type : 3726 Number RLEs to save : 10839 TO DO : save crop sub photo not yet done ! save time : 0.6624102592468262 nb_obj : 6 nb_hashtags : 3 time to prepare the origin masks : 9.799344539642334 time for calcul the mask position with numpy : 0.9724690914154053 nb_pixel_total : 10204293 time to create 1 rle with new method : 0.6567032337188721 time for calcul the mask position with numpy : 0.04834461212158203 nb_pixel_total : 474485 time to create 1 rle with new method : 1.05802583694458 time for calcul the mask position with numpy : 0.044799089431762695 nb_pixel_total : 52650 time to create 1 rle with old method : 0.06499481201171875 time for calcul the mask position with numpy : 0.06441140174865723 nb_pixel_total : 34851 time to create 1 rle with old method : 0.04033350944519043 time for calcul the mask position with numpy : 0.06107974052429199 nb_pixel_total : 112910 time to create 1 rle with old method : 0.1295790672302246 time for calcul the mask position with numpy : 0.07571816444396973 nb_pixel_total : 71001 time to create 1 rle with old method : 0.08306217193603516 time for calcul the mask position with numpy : 0.07381606101989746 nb_pixel_total : 1380050 time to create 1 rle with new method : 0.9546797275543213 create new chi : 4.508283853530884 time to delete rle : 0.0014317035675048828 batch 1 Loaded 7 chid ids of type : 3726 Number RLEs to save : 12462 TO DO : save crop sub photo not yet done ! save time : 0.7724597454071045 nb_obj : 11 nb_hashtags : 4 time to prepare the origin masks : 13.74718976020813 time for calcul the mask position with numpy : 1.1980640888214111 nb_pixel_total : 9185630 time to create 1 rle with new method : 1.148984670639038 time for calcul the mask position with numpy : 0.0671381950378418 nb_pixel_total : 5657 time to create 1 rle with old method : 0.007450103759765625 time for calcul the mask position with numpy : 0.07317066192626953 nb_pixel_total : 464549 time to create 1 rle with new method : 1.0308420658111572 time for calcul the mask position with numpy : 0.06639814376831055 nb_pixel_total : 50122 time to create 1 rle with old method : 0.058408260345458984 time for calcul the mask position with numpy : 0.0663762092590332 nb_pixel_total : 198721 time to create 1 rle with new method : 0.8433349132537842 time for calcul the mask position with numpy : 0.06348156929016113 nb_pixel_total : 197184 time to create 1 rle with new method : 0.8710834980010986 time for calcul the mask position with numpy : 0.07976055145263672 nb_pixel_total : 389307 time to create 1 rle with new method : 1.0380282402038574 time for calcul the mask position with numpy : 0.06531906127929688 nb_pixel_total : 191473 time to create 1 rle with new method : 0.8836824893951416 time for calcul the mask position with numpy : 0.08806848526000977 nb_pixel_total : 1531370 time to create 1 rle with new method : 1.4983136653900146 time for calcul the mask position with numpy : 0.06383109092712402 nb_pixel_total : 10588 time to create 1 rle with old method : 0.011306047439575195 time for calcul the mask position with numpy : 0.07374286651611328 nb_pixel_total : 75582 time to create 1 rle with old method : 0.08443093299865723 time for calcul the mask position with numpy : 0.07793188095092773 nb_pixel_total : 30057 time to create 1 rle with old method : 0.03417158126831055 create new chi : 9.843333721160889 time to delete rle : 0.0027420520782470703 batch 1 Loaded 12 chid ids of type : 3726 Number RLEs to save : 16697 TO DO : save crop sub photo not yet done ! save time : 1.07509446144104 nb_obj : 6 nb_hashtags : 3 time to prepare the origin masks : 9.623579740524292 time for calcul the mask position with numpy : 0.49527978897094727 nb_pixel_total : 9781620 time to create 1 rle with new method : 0.7408890724182129 time for calcul the mask position with numpy : 0.08041572570800781 nb_pixel_total : 135524 time to create 1 rle with old method : 0.17008256912231445 time for calcul the mask position with numpy : 0.042830705642700195 nb_pixel_total : 196424 time to create 1 rle with new method : 0.8327834606170654 time for calcul the mask position with numpy : 0.06759095191955566 nb_pixel_total : 1637728 time to create 1 rle with new method : 0.7264039516448975 time for calcul the mask position with numpy : 0.04048609733581543 nb_pixel_total : 146804 time to create 1 rle with old method : 0.17147254943847656 time for calcul the mask position with numpy : 0.0461728572845459 nb_pixel_total : 79149 time to create 1 rle with old method : 0.09435391426086426 time for calcul the mask position with numpy : 0.04358935356140137 nb_pixel_total : 352991 time to create 1 rle with new method : 0.6581900119781494 create new chi : 4.415045261383057 time to delete rle : 0.0021572113037109375 batch 1 Loaded 7 chid ids of type : 3726 Number RLEs to save : 14494 TO DO : save crop sub photo not yet done ! save time : 0.8946003913879395 nb_obj : 7 nb_hashtags : 3 time to prepare the origin masks : 9.709096908569336 time for calcul the mask position with numpy : 0.81081223487854 nb_pixel_total : 10488620 time to create 1 rle with new method : 1.1196565628051758 time for calcul the mask position with numpy : 0.08471989631652832 nb_pixel_total : 7874 time to create 1 rle with old method : 0.009464025497436523 time for calcul the mask position with numpy : 0.08396387100219727 nb_pixel_total : 92289 time to create 1 rle with old method : 0.14302468299865723 time for calcul the mask position with numpy : 0.04674386978149414 nb_pixel_total : 4494 time to create 1 rle with old method : 0.007038116455078125 time for calcul the mask position with numpy : 0.04658055305480957 nb_pixel_total : 52622 time to create 1 rle with old method : 0.0622563362121582 time for calcul the mask position with numpy : 0.0437467098236084 nb_pixel_total : 45754 time to create 1 rle with old method : 0.05459928512573242 time for calcul the mask position with numpy : 0.04665732383728027 nb_pixel_total : 40454 time to create 1 rle with old method : 0.04866957664489746 time for calcul the mask position with numpy : 0.05323028564453125 nb_pixel_total : 1598133 time to create 1 rle with new method : 0.7068212032318115 create new chi : 3.696425676345825 time to delete rle : 0.0013110637664794922 batch 1 Loaded 8 chid ids of type : 3726 Number RLEs to save : 11708 TO DO : save crop sub photo not yet done ! save time : 0.7207748889923096 nb_obj : 6 nb_hashtags : 4 time to prepare the origin masks : 8.025296688079834 time for calcul the mask position with numpy : 1.060805082321167 nb_pixel_total : 8320643 time to create 1 rle with new method : 2.2720911502838135 time for calcul the mask position with numpy : 0.05962538719177246 nb_pixel_total : 237868 time to create 1 rle with new method : 1.4386589527130127 time for calcul the mask position with numpy : 0.059734344482421875 nb_pixel_total : 85784 time to create 1 rle with old method : 0.1013181209564209 time for calcul the mask position with numpy : 0.07207918167114258 nb_pixel_total : 489513 time to create 1 rle with new method : 0.9060125350952148 time for calcul the mask position with numpy : 0.07271337509155273 nb_pixel_total : 594966 time to create 1 rle with new method : 1.1803674697875977 time for calcul the mask position with numpy : 0.05567741394042969 nb_pixel_total : 291018 time to create 1 rle with new method : 1.2629363536834717 time for calcul the mask position with numpy : 0.09177136421203613 nb_pixel_total : 2310448 time to create 1 rle with new method : 2.2455532550811768 create new chi : 11.224242210388184 time to delete rle : 0.002536773681640625 batch 1 Loaded 7 chid ids of type : 3726 Number RLEs to save : 16730 TO DO : save crop sub photo not yet done ! save time : 1.4856231212615967 nb_obj : 10 nb_hashtags : 5 time to prepare the origin masks : 16.09773850440979 time for calcul the mask position with numpy : 0.5153844356536865 nb_pixel_total : 8075461 time to create 1 rle with new method : 2.6036314964294434 time for calcul the mask position with numpy : 0.0789194107055664 nb_pixel_total : 1900 time to create 1 rle with old method : 0.002777099609375 time for calcul the mask position with numpy : 0.1663806438446045 nb_pixel_total : 296215 time to create 1 rle with new method : 1.5372636318206787 time for calcul the mask position with numpy : 0.08056426048278809 nb_pixel_total : 122481 time to create 1 rle with old method : 0.13528156280517578 time for calcul the mask position with numpy : 0.0814816951751709 nb_pixel_total : 681646 time to create 1 rle with new method : 1.303375005722046 time for calcul the mask position with numpy : 0.07317686080932617 nb_pixel_total : 182262 time to create 1 rle with new method : 0.9585170745849609 time for calcul the mask position with numpy : 0.06820511817932129 nb_pixel_total : 21531 time to create 1 rle with old method : 0.023268461227416992 time for calcul the mask position with numpy : 0.06667542457580566 nb_pixel_total : 46297 time to create 1 rle with old method : 0.050485849380493164 time for calcul the mask position with numpy : 0.06752204895019531 nb_pixel_total : 40729 time to create 1 rle with old method : 0.043901681900024414 time for calcul the mask position with numpy : 0.07051467895507812 nb_pixel_total : 2646 time to create 1 rle with old method : 0.002933502197265625 time for calcul the mask position with numpy : 0.3509690761566162 nb_pixel_total : 2859072 time to create 1 rle with new method : 1.0979673862457275 create new chi : 9.714204788208008 time to delete rle : 0.0044040679931640625 batch 1 Loaded 11 chid ids of type : 3726 Number RLEs to save : 17532 TO DO : save crop sub photo not yet done ! save time : 1.0901553630828857 nb_obj : 8 nb_hashtags : 4 time to prepare the origin masks : 9.375799417495728 time for calcul the mask position with numpy : 0.7994575500488281 nb_pixel_total : 8879536 time to create 1 rle with new method : 0.8040816783905029 time for calcul the mask position with numpy : 0.05988359451293945 nb_pixel_total : 273979 time to create 1 rle with new method : 1.0781219005584717 time for calcul the mask position with numpy : 0.05610203742980957 nb_pixel_total : 345390 time to create 1 rle with new method : 1.1720526218414307 time for calcul the mask position with numpy : 0.05126142501831055 nb_pixel_total : 3865 time to create 1 rle with old method : 0.004437446594238281 time for calcul the mask position with numpy : 0.04795646667480469 nb_pixel_total : 30830 time to create 1 rle with old method : 0.032547950744628906 time for calcul the mask position with numpy : 0.04374527931213379 nb_pixel_total : 200192 time to create 1 rle with new method : 1.8040204048156738 time for calcul the mask position with numpy : 0.08714079856872559 nb_pixel_total : 234253 time to create 1 rle with new method : 0.8133902549743652 time for calcul the mask position with numpy : 0.06569170951843262 nb_pixel_total : 1699 time to create 1 rle with old method : 0.002069711685180664 time for calcul the mask position with numpy : 0.11140275001525879 nb_pixel_total : 2360496 time to create 1 rle with new method : 1.1343927383422852 create new chi : 8.496175765991211 time to delete rle : 0.002737283706665039 batch 1 Loaded 9 chid ids of type : 3726 Number RLEs to save : 16965 TO DO : save crop sub photo not yet done ! save time : 1.021986484527588 nb_obj : 3 nb_hashtags : 1 time to prepare the origin masks : 4.283773899078369 time for calcul the mask position with numpy : 0.6866118907928467 nb_pixel_total : 12074255 time to create 1 rle with new method : 0.8006036281585693 time for calcul the mask position with numpy : 0.0357666015625 nb_pixel_total : 53527 time to create 1 rle with old method : 0.061490774154663086 time for calcul the mask position with numpy : 0.04661154747009277 nb_pixel_total : 20978 time to create 1 rle with old method : 0.03089118003845215 time for calcul the mask position with numpy : 0.047132253646850586 nb_pixel_total : 181480 time to create 1 rle with new method : 0.9257364273071289 create new chi : 2.7591750621795654 time to delete rle : 0.0010044574737548828 batch 1 Loaded 4 chid ids of type : 3726 Number RLEs to save : 4674 TO DO : save crop sub photo not yet done ! save time : 0.31417417526245117 nb_obj : 6 nb_hashtags : 4 time to prepare the origin masks : 9.056494235992432 time for calcul the mask position with numpy : 0.5082740783691406 nb_pixel_total : 10632016 time to create 1 rle with new method : 0.9219307899475098 time for calcul the mask position with numpy : 0.06090044975280762 nb_pixel_total : 5617 time to create 1 rle with old method : 0.013633966445922852 time for calcul the mask position with numpy : 0.06303024291992188 nb_pixel_total : 56685 time to create 1 rle with old method : 0.08583545684814453 time for calcul the mask position with numpy : 0.05297398567199707 nb_pixel_total : 98608 time to create 1 rle with old method : 0.12091827392578125 time for calcul the mask position with numpy : 0.04573655128479004 nb_pixel_total : 694762 time to create 1 rle with new method : 0.9346146583557129 time for calcul the mask position with numpy : 0.04647111892700195 nb_pixel_total : 291565 time to create 1 rle with new method : 0.9108450412750244 time for calcul the mask position with numpy : 0.07880949974060059 nb_pixel_total : 550987 time to create 1 rle with new method : 1.1502327919006348 create new chi : 5.207460403442383 time to delete rle : 0.001735687255859375 batch 1 Loaded 7 chid ids of type : 3726 Number RLEs to save : 12889 TO DO : save crop sub photo not yet done ! save time : 1.038313865661621 map_output_result : {1385918666: (0.0, 'Should be the crop_list due to order', 0.0), 1385918663: (0.0, 'Should be the crop_list due to order', 0.0), 1385918662: (0.0, 'Should be the crop_list due to order', 0.0), 1385918658: (0.0, 'Should be the crop_list due to order', 0.0), 1385918656: (0.0, 'Should be the crop_list due to order', 0.0), 1385918654: (0.0, 'Should be the crop_list due to order', 0.0), 1385918381: (0.0, 'Should be the crop_list due to order', 0.0), 1385918376: (0.0, 'Should be the crop_list due to order', 0.0), 1385918361: (0.0, 'Should be the crop_list due to order', 0.0), 1385918335: (0.0, 'Should be the crop_list due to order', 0.0), 1385918323: (0.0, 'Should be the crop_list due to order', 0.0), 1385918320: (0.0, 'Should be the crop_list due to order', 0.0), 1385917740: (0.0, 'Should be the crop_list due to order', 0.0), 1385917732: (0.0, 'Should be the crop_list due to order', 0.0), 1385917729: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1385918666, 1385918663, 1385918662, 1385918658, 1385918656, 1385918654, 1385918381, 1385918376, 1385918361, 1385918335, 1385918323, 1385918320, 1385917740, 1385917732, 1385917729] Looping around the photos to save general results len do output : 15 /1385918666.Didn't retrieve data . /1385918663.Didn't retrieve data . /1385918662.Didn't retrieve data . /1385918658.Didn't retrieve data . /1385918656.Didn't retrieve data . /1385918654.Didn't retrieve data . /1385918381.Didn't retrieve data . /1385918376.Didn't retrieve data . /1385918361.Didn't retrieve data . /1385918335.Didn't retrieve data . /1385918323.Didn't retrieve data . /1385918320.Didn't retrieve data . /1385917740.Didn't retrieve data . /1385917732.Didn't retrieve data . /1385917729.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918666', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918663', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918662', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918658', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918656', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918654', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918381', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918376', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918361', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918335', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918323', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918320', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917740', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917732', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917729', None, None, None, None, None, '3772638') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 45 time used for this insertion : 0.016895771026611328 save_final save missing photos in datou_result : time spend for datou_step_exec : 265.32066082954407 time spend to save output : 0.020385026931762695 total time spend for step 8 : 265.34104585647583 step9:crop_condition Fri Sep 26 11:38:57 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 15 ! batch 1 Loaded 127 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 12 About to insert : list_path_to_insert length 12 new photo from crops ! About to upload 12 photos upload in portfolio : 4869462 init cache_photo without model_param we have 12 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758879553_66326 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 12 photos in the portfolio 4869462 time of upload the photos Elapsed time : 4.848921537399292 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758879565_66326 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.2749667167663574 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 10 About to insert : list_path_to_insert length 10 new photo from crops ! About to upload 10 photos upload in portfolio : 4869462 init cache_photo without model_param we have 10 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758879576_66326 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 10 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.143996000289917 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758879584_66326 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.7984433174133301 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1385918666, 1385918663, 1385918662, 1385918658, 1385918656, 1385918654, 1385918381, 1385918376, 1385918361, 1385918335, 1385918323, 1385918320, 1385917740, 1385917732, 1385917729] Looping around the photos to save general results len do output : 28 /1386335060Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335061Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335062Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335064Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335065Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335066Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335068Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335069Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335070Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335072Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335073Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335074Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335152Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335154Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335155Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335156Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335158Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335227Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335229Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335231Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335234Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335235Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335237Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335240Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335241Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335242Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335244Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386335256Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918666', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918663', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918662', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918658', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918656', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918654', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918381', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918376', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918361', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918335', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918323', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918320', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917740', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917732', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917729', None, None, None, None, None, '3772638') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 99 time used for this insertion : 0.06529760360717773 save_final save missing photos in datou_result : time spend for datou_step_exec : 48.737626791000366 time spend to save output : 0.06783723831176758 total time spend for step 9 : 48.805464029312134 step10:ventilate_hashtags_in_portfolio Fri Sep 26 11:39:46 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 27193435 get user id for portfolio 27193435 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27193435 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre_refus','flou','papier','mal_croppe','metal','Teint_Dans_La_Masse','plastique','Carton_gris','Carton_brun','kraft','environnement','cartonnette')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27193435 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre_refus','flou','papier','mal_croppe','metal','Teint_Dans_La_Masse','plastique','Carton_gris','Carton_brun','kraft','environnement','cartonnette')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27193435 AND mptpi.`type`=4417 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre_refus','flou','papier','mal_croppe','metal','Teint_Dans_La_Masse','plastique','Carton_gris','Carton_brun','kraft','environnement','cartonnette')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27193435 AND mptpi.`type`=4417 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre_refus','flou','papier','mal_croppe','metal','Teint_Dans_La_Masse','plastique','Carton_gris','Carton_brun','kraft','environnement','cartonnette')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://marlene.fotonower.com/velours/27273428,27273429,27273430,27273431,27273432,27273433,27273434,27273435,27273436,27273437,27273438,27273439?tags=autre_refus,flou,papier,mal_croppe,metal,Teint_Dans_La_Masse,plastique,Carton_gris,Carton_brun,kraft,environnement,cartonnette Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1385918666, 1385918663, 1385918662, 1385918658, 1385918656, 1385918654, 1385918381, 1385918376, 1385918361, 1385918335, 1385918323, 1385918320, 1385917740, 1385917732, 1385917729] Looping around the photos to save general results len do output : 1 /27193435. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918666', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918663', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918662', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918658', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918656', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918654', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918381', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918376', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918361', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918335', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918323', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918320', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917740', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917732', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917729', None, None, None, None, None, '3772638') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 16 time used for this insertion : 0.015162944793701172 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.230240821838379 time spend to save output : 0.015646934509277344 total time spend for step 10 : 2.2458877563476562 step11:final Fri Sep 26 11:39:48 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1385918666: ('0.11091927628898955',), 1385918663: ('0.11091927628898955',), 1385918662: ('0.11091927628898955',), 1385918658: ('0.11091927628898955',), 1385918656: ('0.11091927628898955',), 1385918654: ('0.11091927628898955',), 1385918381: ('0.11091927628898955',), 1385918376: ('0.11091927628898955',), 1385918361: ('0.11091927628898955',), 1385918335: ('0.11091927628898955',), 1385918323: ('0.11091927628898955',), 1385918320: ('0.11091927628898955',), 1385917740: ('0.11091927628898955',), 1385917732: ('0.11091927628898955',), 1385917729: ('0.11091927628898955',)} new output for save of step final : {1385918666: ('0.11091927628898955',), 1385918663: ('0.11091927628898955',), 1385918662: ('0.11091927628898955',), 1385918658: ('0.11091927628898955',), 1385918656: ('0.11091927628898955',), 1385918654: ('0.11091927628898955',), 1385918381: ('0.11091927628898955',), 1385918376: ('0.11091927628898955',), 1385918361: ('0.11091927628898955',), 1385918335: ('0.11091927628898955',), 1385918323: ('0.11091927628898955',), 1385918320: ('0.11091927628898955',), 1385917740: ('0.11091927628898955',), 1385917732: ('0.11091927628898955',), 1385917729: ('0.11091927628898955',)} [1385918666, 1385918663, 1385918662, 1385918658, 1385918656, 1385918654, 1385918381, 1385918376, 1385918361, 1385918335, 1385918323, 1385918320, 1385917740, 1385917732, 1385917729] Looping around the photos to save general results len do output : 15 /1385918666.Didn't retrieve data . /1385918663.Didn't retrieve data . /1385918662.Didn't retrieve data . /1385918658.Didn't retrieve data . /1385918656.Didn't retrieve data . /1385918654.Didn't retrieve data . /1385918381.Didn't retrieve data . /1385918376.Didn't retrieve data . /1385918361.Didn't retrieve data . /1385918335.Didn't retrieve data . /1385918323.Didn't retrieve data . /1385918320.Didn't retrieve data . /1385917740.Didn't retrieve data . /1385917732.Didn't retrieve data . /1385917729.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918666', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918663', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918662', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918658', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918656', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918654', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918381', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918376', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918361', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918335', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918323', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918320', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917740', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917732', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917729', None, None, None, None, None, '3772638') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 45 time used for this insertion : 0.01450657844543457 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.14722537994384766 time spend to save output : 0.015704631805419922 total time spend for step 11 : 0.16293001174926758 step12:velours_tree Fri Sep 26 11:39:48 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.25344133377075195 time spend to save output : 4.38690185546875e-05 total time spend for step 12 : 0.25348520278930664 step13:send_mail_cod Fri Sep 26 11:39:48 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin in order to get the selector url, please entre the license of selector results_Auto_P27193435_26-09-2025_11_39_48.pdf 27273407 imagette272734071758879588 27273409 imagette272734091758879588 27273411 imagette272734111758879588 27273413 imagette272734131758879588 27273415 change filename to text .imagette272734151758879588 27273419 imagette272734191758879589 27273422 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272734221758879589 27273414 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272734141758879589 27273416 imagette272734161758879590 27273418 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272734181758879590 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=27193435 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://marlene.fotonower.com/velours/27273428,27273429,27273430,27273431,27273432,27273433,27273434,27273435,27273436,27273437,27273438,27273439?tags=autre_refus,flou,papier,mal_croppe,metal,Teint_Dans_La_Masse,plastique,Carton_gris,Carton_brun,kraft,environnement,cartonnette your option no_mail is active, we will not send the real mail to your client args[1385918666] : ((1385918666, 0.0036823414631721708, 2107752395), (1385918666, 0.19278024592505497, 492688767), '0.11091927628898955') We are sending mail with results at report@fotonower.com args[1385918663] : ((1385918663, 0.034549027455518036, 2107752395), (1385918663, 0.11011084547759217, 492688767), '0.11091927628898955') We are sending mail with results at report@fotonower.com args[1385918662] : ((1385918662, -0.18643456305444578, 496442774), (1385918662, 0.3851614687982524, 492688767), '0.11091927628898955') We are sending mail with results at report@fotonower.com args[1385918658] : ((1385918658, 0.08131885859372964, 2107752395), (1385918658, 0.4648148929826399, 492688767), '0.11091927628898955') We are sending mail with results at report@fotonower.com args[1385918656] : ((1385918656, 0.12231717180454152, 2107752395), (1385918656, 0.4366129658881093, 492688767), '0.11091927628898955') We are sending mail with results at report@fotonower.com args[1385918654] : ((1385918654, -0.03307886504068204, 2107752395), (1385918654, -0.038099871318829415, 492688767), '0.11091927628898955') We are sending mail with results at report@fotonower.com args[1385918381] : ((1385918381, -0.2809970114282817, 496442774), (1385918381, -0.08043742140382161, 492688767), '0.11091927628898955') We are sending mail with results at report@fotonower.com args[1385918376] : ((1385918376, -0.1987447159273796, 496442774), (1385918376, -0.20958555245810098, 492688767), '0.11091927628898955') We are sending mail with results at report@fotonower.com args[1385918361] : ((1385918361, 0.0013312018998267638, 2107752395), (1385918361, 0.1427283774103872, 492688767), '0.11091927628898955') We are sending mail with results at report@fotonower.com args[1385918335] : ((1385918335, 0.012110414235075159, 2107752395), (1385918335, -0.2700593440680864, 492688767), '0.11091927628898955') We are sending mail with results at report@fotonower.com args[1385918323] : ((1385918323, 0.05978411782740377, 2107752395), (1385918323, 0.40758924724434176, 492688767), '0.11091927628898955') We are sending mail with results at report@fotonower.com args[1385918320] : ((1385918320, -0.2677121497002332, 496442774), (1385918320, 0.5596086512127206, 492688767), '0.11091927628898955') We are sending mail with results at report@fotonower.com args[1385917740] : ((1385917740, -0.3847085937536471, 496442774), (1385917740, -0.262324382504418, 492688767), '0.11091927628898955') We are sending mail with results at report@fotonower.com args[1385917732] : ((1385917732, -0.2911322831716602, 496442774), (1385917732, 0.2753493946975951, 492688767), '0.11091927628898955') We are sending mail with results at report@fotonower.com args[1385917729] : ((1385917729, -0.3309808406605824, 496442774), (1385917729, 0.024410769871462765, 492688767), '0.11091927628898955') We are sending mail with results at report@fotonower.com refus_total : 0.11091927628898955 2022-04-13 10:29:59 0 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27193435_26-09-2025_11_39_48.pdf results_Auto_P27193435_26-09-2025_11_39_48.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27193435_26-09-2025_11_39_48.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4433','27193435','results_Auto_P27193435_26-09-2025_11_39_48.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27193435_26-09-2025_11_39_48.pdf','pdf','','1.0','0.11091927628898955') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1385918666, 1385918663, 1385918662, 1385918658, 1385918656, 1385918654, 1385918381, 1385918376, 1385918361, 1385918335, 1385918323, 1385918320, 1385917740, 1385917732, 1385917729] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918666', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918663', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918662', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918658', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918656', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918654', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918381', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918376', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918361', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918335', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918323', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918320', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917740', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917732', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917729', None, None, None, None, None, '3772638') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 15 time used for this insertion : 0.014796257019042969 save_final save missing photos in datou_result : time spend for datou_step_exec : 8.45463490486145 time spend to save output : 0.015395641326904297 total time spend for step 13 : 8.470030546188354 step14:split_time_score Fri Sep 26 11:39:57 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('10', 30),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 24092025 27193435 Nombre de photos uploadées : 30 / 23040 (0%) 24092025 27193435 Nombre de photos taguées (types de déchets): 0 / 30 (0%) 24092025 27193435 Nombre de photos taguées (volume) : 0 / 30 (0%) elapsed_time : load_data_split_time_score 1.9073486328125e-06 elapsed_time : order_list_meta_photo_and_scores 5.245208740234375e-06 ?????????????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0015475749969482422 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.22217082977294922 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27193431 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27193432 order by id desc limit 1 Qualite : 0.0994719176427741 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27193433_26-09-2025_11_39_42.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27193433 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27193433 AND mptpi.`type`=3726 To do Qualite : 0.11091927628898955 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27193435_26-09-2025_11_39_48.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27193435 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27193435 AND mptpi.`type`=3726 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'24092025': {'nb_upload': 30, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1385918666, 1385918663, 1385918662, 1385918658, 1385918656, 1385918654, 1385918381, 1385918376, 1385918361, 1385918335, 1385918323, 1385918320, 1385917740, 1385917732, 1385917729] Looping around the photos to save general results len do output : 1 /27193435Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918666', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918663', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918662', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918658', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918656', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918654', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918381', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918376', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918361', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918335', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918323', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385918320', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917740', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917732', None, None, None, None, None, '3772638') ('4433', None, None, None, None, None, None, None, '3772638') ('4433', '27193435', '1385917729', None, None, None, None, None, '3772638') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 16 time used for this insertion : 0.014719724655151367 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.435964584350586 time spend to save output : 0.01501321792602539 total time spend for step 14 : 2.4509778022766113 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 376.08user 320.02system 13:03.45elapsed 88%CPU (0avgtext+0avgdata 4648000maxresident)k 8658216inputs+394344outputs (366497major+32170978minor)pagefaults 0swaps