python /home/admin/mtr/script_for_cron.py -j default -m 20 -a 'python3 ~/workarea/git/Velours/python/prod/datou.py -j batch_current -a 4433' -s traitement_sts -M 0 -S 0 -U 100,80,95 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/home/admin/workarea/git/apy', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 4182383 load datou : 4433 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4433, datou_cur_ids : ['3784202'] with mtr_portfolio_ids : ['27268665'] and first list_photo_ids : [] new path : /proc/4182383/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, brightness, blur_detection, crop_condition, thcl, tfhub_classification2, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 15 list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 15 ; length of list_pids : 15 ; length of list_args : 15 time to download the photos : 5.3978917598724365 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 14 step1:mask_detect Fri Sep 26 10:26:54 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10582 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-09-26 10:26:57.831996: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-26 10:26:57.864493: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-26 10:26:57.866368: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fa684000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-26 10:26:57.866419: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-26 10:26:57.869928: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-26 10:26:58.027947: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xef384b0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-26 10:26:58.028505: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-26 10:26:58.029696: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 10:26:58.030303: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 10:26:58.032449: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 10:26:58.050069: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 10:26:58.051096: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 10:26:58.080485: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 10:26:58.084707: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 10:26:58.134478: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 10:26:58.135960: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 10:26:58.136373: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 10:26:58.137303: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-26 10:26:58.137319: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-26 10:26:58.137328: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-26 10:26:58.139252: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9801 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-09-26 10:26:58.419329: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 10:26:58.419442: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 10:26:58.419459: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 10:26:58.419474: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 10:26:58.419488: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 10:26:58.419501: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 10:26:58.419515: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 10:26:58.419529: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 10:26:58.420699: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 10:26:58.421944: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 10:26:58.421973: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 10:26:58.421987: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 10:26:58.422000: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 10:26:58.422013: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 10:26:58.422025: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 10:26:58.422038: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 10:26:58.422051: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 10:26:58.423193: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 10:26:58.423244: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-26 10:26:58.423253: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-26 10:26:58.423261: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-26 10:26:58.424468: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9801 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-09-26 10:27:12.353403: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 10:27:12.636180: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 15 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 18 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 1.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 13 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 5.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 11 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 21 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 7.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 16 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 18 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 22 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 6 Detection mask done ! Trying to reset tf kernel 4182474 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5290 tf kernel not reseted sub process len(results) : 15 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 15 len(list_Values) 0 process is alive process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10582 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.0015063285827636719 nb_pixel_total : 36956 time to create 1 rle with old method : 0.044736623764038086 length of segment : 203 time for calcul the mask position with numpy : 0.0003466606140136719 nb_pixel_total : 15842 time to create 1 rle with old method : 0.018490076065063477 length of segment : 211 time for calcul the mask position with numpy : 0.004121303558349609 nb_pixel_total : 207938 time to create 1 rle with new method : 0.010556936264038086 length of segment : 660 time for calcul the mask position with numpy : 0.00024271011352539062 nb_pixel_total : 10383 time to create 1 rle with old method : 0.011953592300415039 length of segment : 174 time for calcul the mask position with numpy : 0.0009541511535644531 nb_pixel_total : 65872 time to create 1 rle with old method : 0.07292032241821289 length of segment : 352 time for calcul the mask position with numpy : 0.0065729618072509766 nb_pixel_total : 360490 time to create 1 rle with new method : 0.24373388290405273 length of segment : 908 time for calcul the mask position with numpy : 0.0013070106506347656 nb_pixel_total : 87426 time to create 1 rle with old method : 0.0930945873260498 length of segment : 459 time for calcul the mask position with numpy : 0.0436398983001709 nb_pixel_total : 1121943 time to create 1 rle with new method : 0.6843984127044678 length of segment : 2193 time for calcul the mask position with numpy : 0.004778385162353516 nb_pixel_total : 299026 time to create 1 rle with new method : 0.012900829315185547 length of segment : 809 time for calcul the mask position with numpy : 0.00014591217041015625 nb_pixel_total : 6756 time to create 1 rle with old method : 0.008073568344116211 length of segment : 73 time for calcul the mask position with numpy : 0.00510716438293457 nb_pixel_total : 363309 time to create 1 rle with new method : 0.012994766235351562 length of segment : 763 time for calcul the mask position with numpy : 0.028568029403686523 nb_pixel_total : 1053619 time to create 1 rle with new method : 0.33239316940307617 length of segment : 2408 time for calcul the mask position with numpy : 0.002294301986694336 nb_pixel_total : 115600 time to create 1 rle with old method : 0.12377285957336426 length of segment : 325 time for calcul the mask position with numpy : 0.0412747859954834 nb_pixel_total : 314583 time to create 1 rle with new method : 0.588266134262085 length of segment : 1652 time for calcul the mask position with numpy : 0.0003554821014404297 nb_pixel_total : 13367 time to create 1 rle with old method : 0.014972686767578125 length of segment : 128 time for calcul the mask position with numpy : 0.0008311271667480469 nb_pixel_total : 55670 time to create 1 rle with old method : 0.0666360855102539 length of segment : 381 time for calcul the mask position with numpy : 0.0025641918182373047 nb_pixel_total : 155142 time to create 1 rle with new method : 0.00794363021850586 length of segment : 466 time for calcul the mask position with numpy : 0.00020503997802734375 nb_pixel_total : 9321 time to create 1 rle with old method : 0.010684490203857422 length of segment : 68 time for calcul the mask position with numpy : 0.0009741783142089844 nb_pixel_total : 50387 time to create 1 rle with old method : 0.05426383018493652 length of segment : 261 time for calcul the mask position with numpy : 0.006535530090332031 nb_pixel_total : 376804 time to create 1 rle with new method : 0.018557310104370117 length of segment : 704 time for calcul the mask position with numpy : 0.0003638267517089844 nb_pixel_total : 18365 time to create 1 rle with old method : 0.019971847534179688 length of segment : 140 time for calcul the mask position with numpy : 0.0006284713745117188 nb_pixel_total : 42041 time to create 1 rle with old method : 0.04556608200073242 length of segment : 259 time for calcul the mask position with numpy : 0.0001742839813232422 nb_pixel_total : 4433 time to create 1 rle with old method : 0.004999399185180664 length of segment : 117 time for calcul the mask position with numpy : 0.02469944953918457 nb_pixel_total : 1059436 time to create 1 rle with new method : 0.07174205780029297 length of segment : 1227 time for calcul the mask position with numpy : 0.02707648277282715 nb_pixel_total : 940852 time to create 1 rle with new method : 0.08669066429138184 length of segment : 2082 time for calcul the mask position with numpy : 0.0017409324645996094 nb_pixel_total : 49049 time to create 1 rle with old method : 0.06688332557678223 length of segment : 265 time for calcul the mask position with numpy : 0.004901409149169922 nb_pixel_total : 201147 time to create 1 rle with new method : 0.0069577693939208984 length of segment : 342 time for calcul the mask position with numpy : 0.0015513896942138672 nb_pixel_total : 105109 time to create 1 rle with old method : 0.12065887451171875 length of segment : 409 time for calcul the mask position with numpy : 0.0008213520050048828 nb_pixel_total : 26122 time to create 1 rle with old method : 0.04329395294189453 length of segment : 139 time for calcul the mask position with numpy : 0.005255699157714844 nb_pixel_total : 241121 time to create 1 rle with new method : 0.010772228240966797 length of segment : 664 time for calcul the mask position with numpy : 0.002118825912475586 nb_pixel_total : 88723 time to create 1 rle with old method : 0.09819960594177246 length of segment : 303 time for calcul the mask position with numpy : 0.0055370330810546875 nb_pixel_total : 293602 time to create 1 rle with new method : 0.007097005844116211 length of segment : 479 time for calcul the mask position with numpy : 0.025977373123168945 nb_pixel_total : 1109312 time to create 1 rle with new method : 0.14168262481689453 length of segment : 2368 time for calcul the mask position with numpy : 0.0030791759490966797 nb_pixel_total : 214532 time to create 1 rle with new method : 0.007830142974853516 length of segment : 587 time for calcul the mask position with numpy : 0.0007002353668212891 nb_pixel_total : 26804 time to create 1 rle with old method : 0.03037285804748535 length of segment : 203 time for calcul the mask position with numpy : 0.0014576911926269531 nb_pixel_total : 25861 time to create 1 rle with old method : 0.030048131942749023 length of segment : 367 time for calcul the mask position with numpy : 0.0004773139953613281 nb_pixel_total : 10723 time to create 1 rle with old method : 0.012831687927246094 length of segment : 148 time for calcul the mask position with numpy : 0.02218770980834961 nb_pixel_total : 67038 time to create 1 rle with old method : 0.0769200325012207 length of segment : 611 time for calcul the mask position with numpy : 0.00035190582275390625 nb_pixel_total : 17775 time to create 1 rle with old method : 0.021225929260253906 length of segment : 189 time for calcul the mask position with numpy : 0.0066967010498046875 nb_pixel_total : 333318 time to create 1 rle with new method : 0.0504300594329834 length of segment : 786 time for calcul the mask position with numpy : 0.0003752708435058594 nb_pixel_total : 7727 time to create 1 rle with old method : 0.009369373321533203 length of segment : 100 time for calcul the mask position with numpy : 0.002441883087158203 nb_pixel_total : 100874 time to create 1 rle with old method : 0.11377596855163574 length of segment : 409 time for calcul the mask position with numpy : 0.004316806793212891 nb_pixel_total : 211980 time to create 1 rle with new method : 0.007376194000244141 length of segment : 574 time for calcul the mask position with numpy : 0.0004372596740722656 nb_pixel_total : 12224 time to create 1 rle with old method : 0.014127969741821289 length of segment : 142 time for calcul the mask position with numpy : 0.002393007278442383 nb_pixel_total : 82001 time to create 1 rle with old method : 0.09129047393798828 length of segment : 411 time for calcul the mask position with numpy : 0.006358146667480469 nb_pixel_total : 289383 time to create 1 rle with new method : 0.009832382202148438 length of segment : 678 time for calcul the mask position with numpy : 0.013011455535888672 nb_pixel_total : 608491 time to create 1 rle with new method : 0.02582240104675293 length of segment : 1581 time for calcul the mask position with numpy : 0.009496688842773438 nb_pixel_total : 559605 time to create 1 rle with new method : 0.02650594711303711 length of segment : 1301 time for calcul the mask position with numpy : 0.0014767646789550781 nb_pixel_total : 45839 time to create 1 rle with old method : 0.05185222625732422 length of segment : 400 time for calcul the mask position with numpy : 0.0053446292877197266 nb_pixel_total : 207674 time to create 1 rle with new method : 0.012039899826049805 length of segment : 581 time for calcul the mask position with numpy : 0.0028946399688720703 nb_pixel_total : 123495 time to create 1 rle with old method : 0.1401214599609375 length of segment : 469 time for calcul the mask position with numpy : 0.003740072250366211 nb_pixel_total : 179328 time to create 1 rle with new method : 0.0064775943756103516 length of segment : 629 time for calcul the mask position with numpy : 0.004987478256225586 nb_pixel_total : 298313 time to create 1 rle with new method : 0.014927864074707031 length of segment : 615 time for calcul the mask position with numpy : 0.003975629806518555 nb_pixel_total : 143557 time to create 1 rle with old method : 0.1779041290283203 length of segment : 477 time for calcul the mask position with numpy : 0.0006732940673828125 nb_pixel_total : 29805 time to create 1 rle with old method : 0.03392338752746582 length of segment : 155 time for calcul the mask position with numpy : 0.030737638473510742 nb_pixel_total : 1476011 time to create 1 rle with new method : 0.25334715843200684 length of segment : 2304 time for calcul the mask position with numpy : 0.14410972595214844 nb_pixel_total : 2323433 time to create 1 rle with new method : 0.23109197616577148 length of segment : 3026 time for calcul the mask position with numpy : 0.003050088882446289 nb_pixel_total : 135238 time to create 1 rle with old method : 0.15089797973632812 length of segment : 614 time for calcul the mask position with numpy : 0.0006761550903320312 nb_pixel_total : 32071 time to create 1 rle with old method : 0.03646039962768555 length of segment : 166 time for calcul the mask position with numpy : 0.005518198013305664 nb_pixel_total : 147839 time to create 1 rle with old method : 0.16478896141052246 length of segment : 626 time for calcul the mask position with numpy : 0.02635669708251953 nb_pixel_total : 1232781 time to create 1 rle with new method : 0.10199546813964844 length of segment : 1485 time for calcul the mask position with numpy : 0.003017902374267578 nb_pixel_total : 76010 time to create 1 rle with old method : 0.0878443717956543 length of segment : 392 time for calcul the mask position with numpy : 0.028241395950317383 nb_pixel_total : 1240269 time to create 1 rle with new method : 0.1763303279876709 length of segment : 1469 time for calcul the mask position with numpy : 0.0013089179992675781 nb_pixel_total : 24841 time to create 1 rle with old method : 0.028833627700805664 length of segment : 338 time for calcul the mask position with numpy : 0.04028487205505371 nb_pixel_total : 1646417 time to create 1 rle with new method : 0.21819019317626953 length of segment : 2985 time for calcul the mask position with numpy : 0.1074821949005127 nb_pixel_total : 2168935 time to create 1 rle with new method : 0.6398971080780029 length of segment : 3745 time for calcul the mask position with numpy : 0.0009937286376953125 nb_pixel_total : 23113 time to create 1 rle with old method : 0.027303218841552734 length of segment : 319 time for calcul the mask position with numpy : 0.0016150474548339844 nb_pixel_total : 81698 time to create 1 rle with old method : 0.09327197074890137 length of segment : 342 time for calcul the mask position with numpy : 0.00857686996459961 nb_pixel_total : 397440 time to create 1 rle with new method : 0.015258073806762695 length of segment : 835 time for calcul the mask position with numpy : 0.0007867813110351562 nb_pixel_total : 76487 time to create 1 rle with old method : 0.08737802505493164 length of segment : 274 time for calcul the mask position with numpy : 0.00887608528137207 nb_pixel_total : 440314 time to create 1 rle with new method : 0.01381063461303711 length of segment : 898 time for calcul the mask position with numpy : 0.0003960132598876953 nb_pixel_total : 11186 time to create 1 rle with old method : 0.013157844543457031 length of segment : 115 time for calcul the mask position with numpy : 0.0009293556213378906 nb_pixel_total : 42655 time to create 1 rle with old method : 0.04792928695678711 length of segment : 241 time for calcul the mask position with numpy : 0.031147003173828125 nb_pixel_total : 1256620 time to create 1 rle with new method : 0.21266794204711914 length of segment : 2848 time for calcul the mask position with numpy : 0.0009324550628662109 nb_pixel_total : 62804 time to create 1 rle with old method : 0.06702351570129395 length of segment : 357 time for calcul the mask position with numpy : 0.0033817291259765625 nb_pixel_total : 272444 time to create 1 rle with new method : 0.008980512619018555 length of segment : 652 time for calcul the mask position with numpy : 0.03357291221618652 nb_pixel_total : 421130 time to create 1 rle with new method : 0.35703420639038086 length of segment : 1047 time for calcul the mask position with numpy : 0.012024641036987305 nb_pixel_total : 832567 time to create 1 rle with new method : 0.06021738052368164 length of segment : 1699 time for calcul the mask position with numpy : 0.0006515979766845703 nb_pixel_total : 18804 time to create 1 rle with old method : 0.019757747650146484 length of segment : 203 time for calcul the mask position with numpy : 0.011558294296264648 nb_pixel_total : 798357 time to create 1 rle with new method : 0.061331748962402344 length of segment : 1542 time for calcul the mask position with numpy : 0.0005116462707519531 nb_pixel_total : 12920 time to create 1 rle with old method : 0.01403951644897461 length of segment : 147 time for calcul the mask position with numpy : 0.01199650764465332 nb_pixel_total : 459270 time to create 1 rle with new method : 0.0825045108795166 length of segment : 608 time for calcul the mask position with numpy : 0.005722999572753906 nb_pixel_total : 227552 time to create 1 rle with new method : 0.008111715316772461 length of segment : 559 time for calcul the mask position with numpy : 0.00041985511779785156 nb_pixel_total : 16073 time to create 1 rle with old method : 0.017812013626098633 length of segment : 106 time for calcul the mask position with numpy : 0.01232767105102539 nb_pixel_total : 537576 time to create 1 rle with new method : 0.017142534255981445 length of segment : 1047 time for calcul the mask position with numpy : 0.003862619400024414 nb_pixel_total : 150621 time to create 1 rle with new method : 0.005671024322509766 length of segment : 445 time for calcul the mask position with numpy : 0.03836226463317871 nb_pixel_total : 999938 time to create 1 rle with new method : 0.41620635986328125 length of segment : 2448 time for calcul the mask position with numpy : 0.001779317855834961 nb_pixel_total : 69645 time to create 1 rle with old method : 0.07601213455200195 length of segment : 608 time for calcul the mask position with numpy : 0.004481315612792969 nb_pixel_total : 323658 time to create 1 rle with new method : 0.011594772338867188 length of segment : 891 time for calcul the mask position with numpy : 0.0028383731842041016 nb_pixel_total : 131337 time to create 1 rle with old method : 0.14291977882385254 length of segment : 367 time for calcul the mask position with numpy : 0.0035445690155029297 nb_pixel_total : 144424 time to create 1 rle with old method : 0.15960907936096191 length of segment : 584 time for calcul the mask position with numpy : 0.005154132843017578 nb_pixel_total : 365721 time to create 1 rle with new method : 0.013484954833984375 length of segment : 1024 time for calcul the mask position with numpy : 0.0033414363861083984 nb_pixel_total : 183602 time to create 1 rle with new method : 0.0048370361328125 length of segment : 314 time for calcul the mask position with numpy : 0.008105754852294922 nb_pixel_total : 450541 time to create 1 rle with new method : 0.01929163932800293 length of segment : 553 time for calcul the mask position with numpy : 0.001580953598022461 nb_pixel_total : 55884 time to create 1 rle with old method : 0.05834698677062988 length of segment : 365 time for calcul the mask position with numpy : 0.001125335693359375 nb_pixel_total : 49312 time to create 1 rle with old method : 0.05401206016540527 length of segment : 240 time for calcul the mask position with numpy : 0.0017201900482177734 nb_pixel_total : 90898 time to create 1 rle with old method : 0.09903860092163086 length of segment : 306 time for calcul the mask position with numpy : 0.0003407001495361328 nb_pixel_total : 6600 time to create 1 rle with old method : 0.007805347442626953 length of segment : 188 time for calcul the mask position with numpy : 0.0069735050201416016 nb_pixel_total : 380045 time to create 1 rle with new method : 0.011535882949829102 length of segment : 828 time for calcul the mask position with numpy : 0.009329795837402344 nb_pixel_total : 454869 time to create 1 rle with new method : 0.01786208152770996 length of segment : 1330 time for calcul the mask position with numpy : 0.0011224746704101562 nb_pixel_total : 69089 time to create 1 rle with old method : 0.07257413864135742 length of segment : 201 time for calcul the mask position with numpy : 0.012046575546264648 nb_pixel_total : 620331 time to create 1 rle with new method : 0.05182814598083496 length of segment : 1489 time for calcul the mask position with numpy : 0.0004417896270751953 nb_pixel_total : 14859 time to create 1 rle with old method : 0.016206026077270508 length of segment : 169 time for calcul the mask position with numpy : 0.0017917156219482422 nb_pixel_total : 98105 time to create 1 rle with old method : 0.10367798805236816 length of segment : 346 time for calcul the mask position with numpy : 0.0019555091857910156 nb_pixel_total : 123738 time to create 1 rle with old method : 0.13737797737121582 length of segment : 311 time for calcul the mask position with numpy : 0.0055425167083740234 nb_pixel_total : 352601 time to create 1 rle with new method : 0.009977102279663086 length of segment : 595 time for calcul the mask position with numpy : 0.005078792572021484 nb_pixel_total : 338324 time to create 1 rle with new method : 0.011834144592285156 length of segment : 674 time for calcul the mask position with numpy : 0.0037565231323242188 nb_pixel_total : 215868 time to create 1 rle with new method : 0.005883693695068359 length of segment : 540 time for calcul the mask position with numpy : 0.003014802932739258 nb_pixel_total : 199383 time to create 1 rle with new method : 0.006828784942626953 length of segment : 538 time for calcul the mask position with numpy : 0.0034177303314208984 nb_pixel_total : 210625 time to create 1 rle with new method : 0.008452892303466797 length of segment : 531 time for calcul the mask position with numpy : 0.0028755664825439453 nb_pixel_total : 187410 time to create 1 rle with new method : 0.00767207145690918 length of segment : 522 time for calcul the mask position with numpy : 0.0036537647247314453 nb_pixel_total : 164875 time to create 1 rle with new method : 0.005344867706298828 length of segment : 448 time for calcul the mask position with numpy : 0.0004565715789794922 nb_pixel_total : 14002 time to create 1 rle with old method : 0.015136957168579102 length of segment : 130 time for calcul the mask position with numpy : 0.008070945739746094 nb_pixel_total : 392264 time to create 1 rle with new method : 0.011817455291748047 length of segment : 816 time for calcul the mask position with numpy : 0.006087779998779297 nb_pixel_total : 231688 time to create 1 rle with new method : 0.009188413619995117 length of segment : 806 time for calcul the mask position with numpy : 0.007781028747558594 nb_pixel_total : 315902 time to create 1 rle with new method : 0.012166976928710938 length of segment : 755 time for calcul the mask position with numpy : 0.014149665832519531 nb_pixel_total : 755460 time to create 1 rle with new method : 0.019902706146240234 length of segment : 850 time for calcul the mask position with numpy : 0.002728700637817383 nb_pixel_total : 131630 time to create 1 rle with old method : 0.14555573463439941 length of segment : 567 time for calcul the mask position with numpy : 0.023988962173461914 nb_pixel_total : 508445 time to create 1 rle with new method : 0.16315722465515137 length of segment : 1970 time for calcul the mask position with numpy : 0.001352071762084961 nb_pixel_total : 51765 time to create 1 rle with old method : 0.05578804016113281 length of segment : 309 time for calcul the mask position with numpy : 0.00048232078552246094 nb_pixel_total : 36118 time to create 1 rle with old method : 0.039359092712402344 length of segment : 255 time for calcul the mask position with numpy : 0.0035791397094726562 nb_pixel_total : 272003 time to create 1 rle with new method : 0.009661436080932617 length of segment : 511 time for calcul the mask position with numpy : 0.005650043487548828 nb_pixel_total : 320590 time to create 1 rle with new method : 0.008942842483520508 length of segment : 970 time for calcul the mask position with numpy : 0.03136634826660156 nb_pixel_total : 1632639 time to create 1 rle with new method : 0.12950897216796875 length of segment : 2587 time for calcul the mask position with numpy : 0.03131508827209473 nb_pixel_total : 22473 time to create 1 rle with old method : 0.023572444915771484 length of segment : 407 time for calcul the mask position with numpy : 0.0012538433074951172 nb_pixel_total : 9487 time to create 1 rle with old method : 0.011749267578125 length of segment : 274 time for calcul the mask position with numpy : 0.009014606475830078 nb_pixel_total : 354365 time to create 1 rle with new method : 0.013465166091918945 length of segment : 653 time for calcul the mask position with numpy : 0.00908660888671875 nb_pixel_total : 437368 time to create 1 rle with new method : 0.014238834381103516 length of segment : 655 time for calcul the mask position with numpy : 0.0048601627349853516 nb_pixel_total : 216728 time to create 1 rle with new method : 0.006887197494506836 length of segment : 519 time for calcul the mask position with numpy : 0.0005795955657958984 nb_pixel_total : 17196 time to create 1 rle with old method : 0.018829822540283203 length of segment : 226 time for calcul the mask position with numpy : 0.0012555122375488281 nb_pixel_total : 33734 time to create 1 rle with old method : 0.0367581844329834 length of segment : 268 time for calcul the mask position with numpy : 0.044828176498413086 nb_pixel_total : 1805660 time to create 1 rle with new method : 0.17899656295776367 length of segment : 3610 time for calcul the mask position with numpy : 0.005435943603515625 nb_pixel_total : 396603 time to create 1 rle with new method : 0.01405787467956543 length of segment : 972 time for calcul the mask position with numpy : 0.0014178752899169922 nb_pixel_total : 33438 time to create 1 rle with old method : 0.045014142990112305 length of segment : 310 time for calcul the mask position with numpy : 0.024980783462524414 nb_pixel_total : 1078271 time to create 1 rle with new method : 0.07828116416931152 length of segment : 2345 time for calcul the mask position with numpy : 0.002033710479736328 nb_pixel_total : 102085 time to create 1 rle with old method : 0.1102900505065918 length of segment : 463 time for calcul the mask position with numpy : 0.0001964569091796875 nb_pixel_total : 12001 time to create 1 rle with old method : 0.013603687286376953 length of segment : 99 time for calcul the mask position with numpy : 0.000316619873046875 nb_pixel_total : 16079 time to create 1 rle with old method : 0.01828169822692871 length of segment : 160 time for calcul the mask position with numpy : 0.0012905597686767578 nb_pixel_total : 98949 time to create 1 rle with old method : 0.10751056671142578 length of segment : 472 time for calcul the mask position with numpy : 0.00021791458129882812 nb_pixel_total : 6995 time to create 1 rle with old method : 0.007967472076416016 length of segment : 121 time for calcul the mask position with numpy : 0.00024580955505371094 nb_pixel_total : 15760 time to create 1 rle with old method : 0.01788187026977539 length of segment : 105 time for calcul the mask position with numpy : 0.0010731220245361328 nb_pixel_total : 27050 time to create 1 rle with old method : 0.029927492141723633 length of segment : 221 time for calcul the mask position with numpy : 0.0013308525085449219 nb_pixel_total : 40003 time to create 1 rle with old method : 0.0453643798828125 length of segment : 136 time for calcul the mask position with numpy : 0.0007145404815673828 nb_pixel_total : 22462 time to create 1 rle with old method : 0.02520012855529785 length of segment : 341 time for calcul the mask position with numpy : 0.0031371116638183594 nb_pixel_total : 136636 time to create 1 rle with old method : 0.14427614212036133 length of segment : 577 time for calcul the mask position with numpy : 0.0006103515625 nb_pixel_total : 15247 time to create 1 rle with old method : 0.017303466796875 length of segment : 166 time for calcul the mask position with numpy : 0.0017485618591308594 nb_pixel_total : 64801 time to create 1 rle with old method : 0.07472920417785645 length of segment : 396 time for calcul the mask position with numpy : 0.008991241455078125 nb_pixel_total : 277422 time to create 1 rle with new method : 0.014261245727539062 length of segment : 672 time for calcul the mask position with numpy : 0.0013957023620605469 nb_pixel_total : 52554 time to create 1 rle with old method : 0.05835890769958496 length of segment : 315 time for calcul the mask position with numpy : 0.01413273811340332 nb_pixel_total : 411853 time to create 1 rle with new method : 0.06133246421813965 length of segment : 1088 time for calcul the mask position with numpy : 0.004008054733276367 nb_pixel_total : 155916 time to create 1 rle with new method : 0.005593538284301758 length of segment : 421 time for calcul the mask position with numpy : 0.0071086883544921875 nb_pixel_total : 478716 time to create 1 rle with new method : 0.016717910766601562 length of segment : 921 time for calcul the mask position with numpy : 0.0006365776062011719 nb_pixel_total : 26673 time to create 1 rle with old method : 0.030987024307250977 length of segment : 129 time for calcul the mask position with numpy : 0.003201723098754883 nb_pixel_total : 109226 time to create 1 rle with old method : 0.11893749237060547 length of segment : 391 time for calcul the mask position with numpy : 0.0008733272552490234 nb_pixel_total : 47683 time to create 1 rle with old method : 0.05178070068359375 length of segment : 355 time for calcul the mask position with numpy : 0.0008509159088134766 nb_pixel_total : 16662 time to create 1 rle with old method : 0.018610239028930664 length of segment : 164 time for calcul the mask position with numpy : 0.00906991958618164 nb_pixel_total : 405301 time to create 1 rle with new method : 0.013464689254760742 length of segment : 779 time for calcul the mask position with numpy : 0.007312297821044922 nb_pixel_total : 408257 time to create 1 rle with new method : 0.018004655838012695 length of segment : 737 time for calcul the mask position with numpy : 0.033600568771362305 nb_pixel_total : 839912 time to create 1 rle with new method : 0.14612078666687012 length of segment : 2632 time for calcul the mask position with numpy : 0.002893686294555664 nb_pixel_total : 150820 time to create 1 rle with new method : 0.006693601608276367 length of segment : 559 time for calcul the mask position with numpy : 0.0014548301696777344 nb_pixel_total : 49440 time to create 1 rle with old method : 0.054403066635131836 length of segment : 275 time for calcul the mask position with numpy : 0.10943436622619629 nb_pixel_total : 2208321 time to create 1 rle with new method : 0.3446629047393799 length of segment : 5238 time for calcul the mask position with numpy : 0.00015211105346679688 nb_pixel_total : 6042 time to create 1 rle with old method : 0.006937742233276367 length of segment : 94 time for calcul the mask position with numpy : 0.0002999305725097656 nb_pixel_total : 12358 time to create 1 rle with old method : 0.013734817504882812 length of segment : 105 time for calcul the mask position with numpy : 0.026491165161132812 nb_pixel_total : 1679609 time to create 1 rle with new method : 0.17337560653686523 length of segment : 2445 time for calcul the mask position with numpy : 0.00173187255859375 nb_pixel_total : 139206 time to create 1 rle with old method : 0.14964890480041504 length of segment : 472 time for calcul the mask position with numpy : 0.0018410682678222656 nb_pixel_total : 136967 time to create 1 rle with old method : 0.14965581893920898 length of segment : 519 time spent for convertir_results : 38.44568109512329 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 167 chid ids of type : 3663 Number RLEs to save : 123582 save missing photos in datou_result : time spend for datou_step_exec : 120.35536670684814 time spend to save output : 6.992773056030273 total time spend for step 1 : 127.34813976287842 step2:brightness Fri Sep 26 10:29:01 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1758875208_4182383_1386038142_1fed7805efab59b4f92b5fa032d205f2.jpg treat image : temp/1758875208_4182383_1386038140_94b6bdf6a266ed505777356c952d9cf8.jpg treat image : temp/1758875208_4182383_1386038138_6c6e3b9e5d761766a6b6c3891093cce6.jpg treat image : temp/1758875208_4182383_1386038137_b49dac4e1b4372f5fb7b47eb1b864ba2.jpg treat image : temp/1758875208_4182383_1386038136_cce810f4277a6a6837e6a10f60282eff.jpg treat image : temp/1758875208_4182383_1386037955_f5e351c86db4be7f645acce8e98328fc.jpg treat image : temp/1758875208_4182383_1386037926_3e24ed45a24ab0ad877b20c74c2efeb2.jpg treat image : temp/1758875208_4182383_1386037914_61f14c37b0115a27ad5222cff0059f6a.jpg treat image : temp/1758875208_4182383_1386037894_5087993376f1d446f9f898c01dfb9319.jpg treat image : temp/1758875208_4182383_1386037858_88a1c550e29cca5c5d9af082685efac3.jpg treat image : temp/1758875208_4182383_1386037850_72d5fd97d6245c749bde7c9e20ee2578.jpg treat image : temp/1758875208_4182383_1386035648_f2fa7e98faf6e077d8069ab08845feae.jpg treat image : temp/1758875208_4182383_1386035593_caba9dda06e35c2242371dc55b10b8ba.jpg treat image : temp/1758875208_4182383_1386035560_303933f4322dd0b7e0004f49578b2c98.jpg treat image : temp/1758875208_4182383_1386035526_991d4751f43b97693d1210809b83a299.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 15 time used for this insertion : 0.013668060302734375 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 15 time used for this insertion : 0.014013290405273438 save missing photos in datou_result : time spend for datou_step_exec : 23.562389373779297 time spend to save output : 0.0570836067199707 total time spend for step 2 : 23.619472980499268 step3:blur_detection Fri Sep 26 10:29:25 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1758875208_4182383_1386038142_1fed7805efab59b4f92b5fa032d205f2.jpg resize: (3040, 4056) 1386038142 -0.09091778290924059 treat image : temp/1758875208_4182383_1386038140_94b6bdf6a266ed505777356c952d9cf8.jpg resize: (3040, 4056) 1386038140 -0.7128978207937444 treat image : temp/1758875208_4182383_1386038138_6c6e3b9e5d761766a6b6c3891093cce6.jpg resize: (3040, 4056) 1386038138 -0.00790061244724607 treat image : temp/1758875208_4182383_1386038137_b49dac4e1b4372f5fb7b47eb1b864ba2.jpg resize: (3040, 4056) 1386038137 0.5687747367354268 treat image : temp/1758875208_4182383_1386038136_cce810f4277a6a6837e6a10f60282eff.jpg resize: (3040, 4056) 1386038136 -0.21231373955313973 treat image : temp/1758875208_4182383_1386037955_f5e351c86db4be7f645acce8e98328fc.jpg resize: (3040, 4056) 1386037955 -0.4334229204692662 treat image : temp/1758875208_4182383_1386037926_3e24ed45a24ab0ad877b20c74c2efeb2.jpg resize: (3040, 4056) 1386037926 -0.028383289366514408 treat image : temp/1758875208_4182383_1386037914_61f14c37b0115a27ad5222cff0059f6a.jpg resize: (3040, 4056) 1386037914 -0.1520993329533846 treat image : temp/1758875208_4182383_1386037894_5087993376f1d446f9f898c01dfb9319.jpg resize: (3040, 4056) 1386037894 0.062414320368215766 treat image : temp/1758875208_4182383_1386037858_88a1c550e29cca5c5d9af082685efac3.jpg resize: (3040, 4056) 1386037858 -0.19305770316564158 treat image : temp/1758875208_4182383_1386037850_72d5fd97d6245c749bde7c9e20ee2578.jpg resize: (3040, 4056) 1386037850 0.16683814579050424 treat image : temp/1758875208_4182383_1386035648_f2fa7e98faf6e077d8069ab08845feae.jpg resize: (3040, 4056) 1386035648 0.48755966284071733 treat image : temp/1758875208_4182383_1386035593_caba9dda06e35c2242371dc55b10b8ba.jpg resize: (3040, 4056) 1386035593 -0.0118089054542413 treat image : temp/1758875208_4182383_1386035560_303933f4322dd0b7e0004f49578b2c98.jpg resize: (3040, 4056) 1386035560 -0.12018177399552336 treat image : temp/1758875208_4182383_1386035526_991d4751f43b97693d1210809b83a299.jpg resize: (3040, 4056) 1386035526 0.17590971396293972 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 15 time used for this insertion : 0.02559494972229004 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 15 time used for this insertion : 0.0177309513092041 save missing photos in datou_result : time spend for datou_step_exec : 82.72689723968506 time spend to save output : 0.17026543617248535 total time spend for step 3 : 82.89716267585754 step4:crop_condition Fri Sep 26 10:30:47 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 15 ! batch 1 Loaded 167 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 15 About to insert : list_path_to_insert length 15 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 22 About to insert : list_path_to_insert length 22 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 12 About to insert : list_path_to_insert length 12 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 31 About to insert : list_path_to_insert length 31 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 27 About to insert : list_path_to_insert length 27 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 14 About to insert : list_path_to_insert length 14 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1386038142, 1386038140, 1386038138, 1386038137, 1386038136, 1386037955, 1386037926, 1386037914, 1386037894, 1386037858, 1386037850, 1386035648, 1386035593, 1386035560, 1386035526] Looping around the photos to save general results len do output : 122 /-3976576964Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576966Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576979Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576997Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577009Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577016Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577028Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577032Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577050Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577068Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577079Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577088Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577089Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577092Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577098Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576967Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576970Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576978Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576985Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576989Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577001Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577003Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577010Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577014Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577022Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577030Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577034Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577039Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577044Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577049Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577053Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577054Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577060Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577076Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577110Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577119Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577128Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576988Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577006Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577008Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577013Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577021Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577031Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577037Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577069Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577070Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577097Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577115Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577122Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576968Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576974Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576976Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576982Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576983Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576990Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576991Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576993Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577007Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577011Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577015Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577020Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577036Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577046Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577047Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577051Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577052Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577057Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577065Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577066Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577071Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577072Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577078Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577084Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577094Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577102Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577106Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577112Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577113Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577114Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577127Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576962Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576963Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576972Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576980Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576987Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576996Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576998Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577002Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577004Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577005Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577019Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577033Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577042Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577043Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577045Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577058Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577074Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577075Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577077Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577091Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577095Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577103Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577104Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577105Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577107Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577108Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577111Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576977Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576981Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976576992Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577024Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577041Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577055Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577056Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577062Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577063Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577067Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577073Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577090Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577109Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577118Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976577100Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038142', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038140', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038138', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038137', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038136', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037955', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037926', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037914', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037894', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037858', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037850', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035648', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035593', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035560', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035526', None, None, None, None, None, '3784202') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 381 time used for this insertion : 0.04546236991882324 save_final save missing photos in datou_result : time spend for datou_step_exec : 148.26409888267517 time spend to save output : 0.04995107650756836 total time spend for step 4 : 148.31404995918274 step5:thcl Fri Sep 26 10:33:16 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 1 we are using the classfication for multi_thcl [2456] time to import caffe and check if the image exist : 0.030120372772216797 time to convert the images to numpy array : 1.0231094360351562 time to import caffe and check if the image exist : 0.028042078018188477 time to convert the images to numpy array : 1.0313341617584229 time to import caffe and check if the image exist : 0.03124547004699707 time to convert the images to numpy array : 1.06813645362854 time to import caffe and check if the image exist : 0.03154873847961426 time to convert the images to numpy array : 1.5647306442260742 time to import caffe and check if the image exist : 0.027453899383544922 time to convert the images to numpy array : 1.5929310321807861 time to import caffe and check if the image exist : 0.030794620513916016 time to convert the images to numpy array : 2.0600428581237793 time to import caffe and check if the image exist : 0.012292146682739258 time to convert the images to numpy array : 2.091872215270996 time to import caffe and check if the image exist : 0.028280973434448242 time to convert the images to numpy array : 2.0835320949554443 time to import caffe and check if the image exist : 0.03597140312194824 time to convert the images to numpy array : 2.5074665546417236 time to import caffe and check if the image exist : 0.03573870658874512 time to convert the images to numpy array : 2.5352818965911865 total time to convert the images to numpy array : 3.0682055950164795 list photo_ids error: [] list photo_ids correct : [-3976577073, -3976577090, -3976577109, -3976577118, -3976577100, -3976577114, -3976577127, -3976576962, -3976576963, -3976576972, -3976576980, -3976576987, -3976576996, -3976576998, -3976577002, -3976577004, -3976577005, -3976577019, -3976577033, -3976577042, -3976577043, -3976577045, -3976577058, -3976577074, -3976577075, -3976577077, -3976577091, -3976577095, -3976577103, -3976577104, -3976577105, -3976576982, -3976576983, -3976576990, -3976576991, -3976576993, -3976577007, -3976577011, -3976577015, -3976577020, -3976577036, -3976577046, -3976577047, -3976577051, -3976577008, -3976577013, -3976577021, -3976577031, -3976577037, -3976577069, -3976577070, -3976577097, -3976577115, -3976577122, -3976576968, -3976576974, -3976576976, -3976577034, -3976577039, -3976577044, -3976577049, -3976577053, -3976577054, -3976577060, -3976577076, -3976577110, -3976577119, -3976577128, -3976576988, -3976577006, -3976576964, -3976576966, -3976576979, -3976576997, -3976577009, -3976577016, -3976577028, -3976577032, -3976577050, -3976577068, -3976577079, -3976577088, -3976577089, -3976577052, -3976577057, -3976577065, -3976577066, -3976577071, -3976577072, -3976577078, -3976577084, -3976577094, -3976577102, -3976577106, -3976577112, -3976577113, -3976577092, -3976577098, -3976576967, -3976576970, -3976576978, -3976576985, -3976576989, -3976577001, -3976577003, -3976577010, -3976577014, -3976577022, -3976577030, -3976577107, -3976577108, -3976577111, -3976576977, -3976576981, -3976576992, -3976577024, -3976577041, -3976577055, -3976577056, -3976577062, -3976577063, -3976577067] number of photos to traite : 122 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10582 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/:/home/admin/workarea/git/apy/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10360 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 3.772451877593994 time used to do the prediction : 0.31516313552856445 we don't save the descriptors for this thcl 2456 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 1.5974044799804688e-05 save missing photos in datou_result : time spend for datou_step_exec : 12.29562497138977 time spend to save output : 0.0003273487091064453 total time spend for step 5 : 12.295952320098877 step6:tfhub_classification2 Fri Sep 26 10:33:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou_step TFHub with tf2 ! we are using the classfication for only one thcl 3501 begin to check gpu status inside check gpu memory 2025-09-26 10:33:32.257902: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-26 10:33:32.260110: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 10:33:32.260247: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 10:33:32.260311: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 10:33:32.279858: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 10:33:32.279997: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 10:33:32.313214: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 10:33:32.317921: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 10:33:32.374155: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 10:33:32.376227: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 10:33:32.377107: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-26 10:33:32.408477: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-26 10:33:32.410519: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fa570000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-26 10:33:32.410549: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-26 10:33:32.416006: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x19fda530 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-26 10:33:32.416030: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-26 10:33:32.417346: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 10:33:32.417425: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 10:33:32.417446: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 10:33:32.417542: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 10:33:32.417564: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 10:33:32.417592: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 10:33:32.417621: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 10:33:32.417652: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 10:33:32.419370: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 10:33:32.419792: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 10:33:32.419859: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-26 10:33:32.419873: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-26 10:33:32.419883: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-26 10:33:32.421577: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3096 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) l 3637 free memory gpu now : 10358 max_wait_temp : 1 max_wait : 5 1 Physical GPUs, 1 Logical GPUs tagging for thcl : 3501 To do loadFromThcl(), then load ParamDescType : thcl3501 thcls : [{'id': 3501, 'mtr_user_id': 31, 'name': 'learn_papier_mixte_centre_tf_b4_220223_3', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '9087724,9087727,9087725,9087729,9087726,9087723,9087722,9087720,9087721,9087728', 'photo_hashtag_type': 4539, 'photo_desc_type': 5759, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 3501, 'mtr_user_id': 31, 'name': 'learn_papier_mixte_centre_tf_b4_220223_3', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '9087724,9087727,9087725,9087729,9087726,9087723,9087722,9087720,9087721,9087728', 'photo_hashtag_type': 4539, 'photo_desc_type': 5759, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5759 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5759, 'learn_papier_mixte_centre_tf_b4_220223_3', 1792, 1792, 'learn_papier_mixte_centre_tf_b4_220223_3', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 2, 22, 13, 54, 32), datetime.datetime(2023, 2, 22, 13, 54, 32)) model_name : learn_papier_mixte_centre_tf_b4_220223_3 model_param file didn't exist model_name : learn_papier_mixte_centre_tf_b4_220223_3 model_type : tf_classification2 list file need : ['Confusion_Matrix.png', 'Precision_Recall_Carton_brun.jpg', 'Precision_Recall_Carton_gris.jpg', 'Precision_Recall_Teint_Dans_La_Masse.jpg', 'Precision_Recall_autre_refus.jpg', 'Precision_Recall_cartonnette.jpg', 'Precision_Recall_environnement.jpg', 'Precision_Recall_kraft.jpg', 'Precision_Recall_metal.jpg', 'Precision_Recall_papier.jpg', 'Precision_Recall_plastique.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file exist in s3 : ['Confusion_Matrix.png', 'Precision_Recall_Carton_brun.jpg', 'Precision_Recall_Carton_gris.jpg', 'Precision_Recall_Teint_Dans_La_Masse.jpg', 'Precision_Recall_autre_refus.jpg', 'Precision_Recall_cartonnette.jpg', 'Precision_Recall_environnement.jpg', 'Precision_Recall_kraft.jpg', 'Precision_Recall_metal.jpg', 'Precision_Recall_papier.jpg', 'Precision_Recall_plastique.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file manque in s3 : [] local folder : /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3 /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Confusion_Matrix.png size_local : 121269 size in s3 : 121269 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:17 Confusion_Matrix.png already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_Carton_brun.jpg size_local : 76104 size in s3 : 76104 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_Carton_brun.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_Carton_gris.jpg size_local : 82912 size in s3 : 82912 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_Carton_gris.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_Teint_Dans_La_Masse.jpg size_local : 75423 size in s3 : 75423 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:25 Precision_Recall_Teint_Dans_La_Masse.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_autre_refus.jpg size_local : 78883 size in s3 : 78883 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_autre_refus.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_cartonnette.jpg size_local : 81791 size in s3 : 81791 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:30 Precision_Recall_cartonnette.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_environnement.jpg size_local : 59936 size in s3 : 59936 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:17 Precision_Recall_environnement.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_kraft.jpg size_local : 80280 size in s3 : 80280 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:25 Precision_Recall_kraft.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_metal.jpg size_local : 77736 size in s3 : 77736 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_metal.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_papier.jpg size_local : 69041 size in s3 : 69041 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_papier.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_plastique.jpg size_local : 70770 size in s3 : 70770 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:17 Precision_Recall_plastique.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Result_Summary.txt size_local : 1092 size in s3 : 1092 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Result_Summary.txt already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/checkpoint size_local : 99 size in s3 : 99 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 checkpoint already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_checkpoint.ckpt.data-00000-of-00002 size_local : 429801 size in s3 : 429801 create time local : 2023-03-03 12:26:16 create time in s3 : 2023-02-22 12:54:30 model_checkpoint.ckpt.data-00000-of-00002 already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_checkpoint.ckpt.data-00001-of-00002 size_local : 141033188 size in s3 : 141033188 create time local : 2023-03-03 12:26:17 create time in s3 : 2023-02-22 12:54:25 model_checkpoint.ckpt.data-00001-of-00002 already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_checkpoint.ckpt.index size_local : 85618 size in s3 : 85618 create time local : 2023-03-03 12:26:17 create time in s3 : 2023-02-22 12:54:30 model_checkpoint.ckpt.index already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_weights.h5 size_local : 71289448 size in s3 : 71289448 create time local : 2023-03-03 12:26:18 create time in s3 : 2023-02-22 12:54:17 model_weights.h5 already exist and didn't need to update desc size : 1792 Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= module (KerasLayer) (None, 1792) 17673816 _________________________________________________________________ learn_papier_mixte_centre_tf (None, 10) 17930 ================================================================= Total params: 17,691,746 Trainable params: 17,930 Non-trainable params: 17,673,816 _________________________________________________________________ Loading Weights... time used to create the model : 17.054335355758667 time used to load_weights : 0.30930471420288086 0it [00:00, ?it/s]temp/1758875208_4182383_1386038142_1fed7805efab59b4f92b5fa032d205f2_rle_crop_3976576964_0.png temp/1758875208_4182383_1386038142_1fed7805efab59b4f92b5fa032d205f2_rle_crop_3976576966_0.png temp/1758875208_4182383_1386038140_94b6bdf6a266ed505777356c952d9cf8_rle_crop_3976576979_0.png temp/1758875208_4182383_1386038137_b49dac4e1b4372f5fb7b47eb1b864ba2_rle_crop_3976576997_0.png temp/1758875208_4182383_1386038136_cce810f4277a6a6837e6a10f60282eff_rle_crop_3976577009_0.png temp/1758875208_4182383_1386037955_f5e351c86db4be7f645acce8e98328fc_rle_crop_3976577016_0.png temp/1758875208_4182383_1386037926_3e24ed45a24ab0ad877b20c74c2efeb2_rle_crop_3976577028_0.png temp/1758875208_4182383_1386037914_61f14c37b0115a27ad5222cff0059f6a_rle_crop_3976577032_0.png temp/1758875208_4182383_1386037894_5087993376f1d446f9f898c01dfb9319_rle_crop_3976577050_0.png temp/1758875208_4182383_1386037850_72d5fd97d6245c749bde7c9e20ee2578_rle_crop_3976577068_0.png temp/1758875208_4182383_1386035648_f2fa7e98faf6e077d8069ab08845feae_rle_crop_3976577079_0.png temp/1758875208_4182383_1386035593_caba9dda06e35c2242371dc55b10b8ba_rle_crop_3976577088_0.png temp/1758875208_4182383_1386035593_caba9dda06e35c2242371dc55b10b8ba_rle_crop_3976577089_0.png temp/1758875208_4182383_1386035593_caba9dda06e35c2242371dc55b10b8ba_rle_crop_3976577092_0.png temp/1758875208_4182383_1386035593_caba9dda06e35c2242371dc55b10b8ba_rle_crop_3976577098_0.png temp/1758875208_4182383_1386038142_1fed7805efab59b4f92b5fa032d205f2_rle_crop_3976576967_0.png temp/1758875208_4182383_1386038142_1fed7805efab59b4f92b5fa032d205f2_rle_crop_3976576970_0.png temp/1758875208_4182383_1386038140_94b6bdf6a266ed505777356c952d9cf8_rle_crop_3976576978_0.png temp/1758875208_4182383_1386038140_94b6bdf6a266ed505777356c952d9cf8_rle_crop_3976576985_0.png temp/1758875208_4182383_1386038138_6c6e3b9e5d761766a6b6c3891093cce6_rle_crop_3976576989_0.png temp/1758875208_4182383_1386038137_b49dac4e1b4372f5fb7b47eb1b864ba2_rle_crop_3976577001_0.png temp/1758875208_4182383_1386038136_cce810f4277a6a6837e6a10f60282eff_rle_crop_3976577003_0.png temp/1758875208_4182383_1386038136_cce810f4277a6a6837e6a10f60282eff_rle_crop_3976577010_0.png temp/1758875208_4182383_1386038136_cce810f4277a6a6837e6a10f60282eff_rle_crop_3976577014_0.png temp/1758875208_4182383_1386037955_f5e351c86db4be7f645acce8e98328fc_rle_crop_3976577022_0.png temp/1758875208_4182383_1386037926_3e24ed45a24ab0ad877b20c74c2efeb2_rle_crop_3976577030_0.png temp/1758875208_4182383_1386037914_61f14c37b0115a27ad5222cff0059f6a_rle_crop_3976577034_0.png temp/1758875208_4182383_1386037914_61f14c37b0115a27ad5222cff0059f6a_rle_crop_3976577039_0.png temp/1758875208_4182383_1386037894_5087993376f1d446f9f898c01dfb9319_rle_crop_3976577044_0.png temp/1758875208_4182383_1386037894_5087993376f1d446f9f898c01dfb9319_rle_crop_3976577049_0.png temp/1758875208_4182383_1386037894_5087993376f1d446f9f898c01dfb9319_rle_crop_3976577053_0.png temp/1758875208_4182383_1386037894_5087993376f1d446f9f898c01dfb9319_rle_crop_3976577054_0.png temp/1758875208_4182383_1386037858_88a1c550e29cca5c5d9af082685efac3_rle_crop_3976577060_0.png temp/1758875208_4182383_1386035648_f2fa7e98faf6e077d8069ab08845feae_rle_crop_3976577076_0.png temp/1758875208_4182383_1386035560_303933f4322dd0b7e0004f49578b2c98_rle_crop_3976577110_0.png temp/1758875208_4182383_1386035560_303933f4322dd0b7e0004f49578b2c98_rle_crop_3976577119_0.png temp/1758875208_4182383_1386035526_991d4751f43b97693d1210809b83a299_rle_crop_3976577128_0.png temp/1758875208_4182383_1386038138_6c6e3b9e5d761766a6b6c3891093cce6_rle_crop_3976576988_0.png temp/1758875208_4182383_1386038136_cce810f4277a6a6837e6a10f60282eff_rle_crop_3976577006_0.png temp/1758875208_4182383_1386038136_cce810f4277a6a6837e6a10f60282eff_rle_crop_3976577008_0.png temp/1758875208_4182383_1386038136_cce810f4277a6a6837e6a10f60282eff_rle_crop_3976577013_0.png temp/1758875208_4182383_1386037955_f5e351c86db4be7f645acce8e98328fc_rle_crop_3976577021_0.png temp/1758875208_4182383_1386037926_3e24ed45a24ab0ad877b20c74c2efeb2_rle_crop_3976577031_0.png temp/1758875208_4182383_1386037914_61f14c37b0115a27ad5222cff0059f6a_rle_crop_3976577037_0.png temp/1758875208_4182383_1386037850_72d5fd97d6245c749bde7c9e20ee2578_rle_crop_3976577069_0.png temp/1758875208_4182383_1386037850_72d5fd97d6245c749bde7c9e20ee2578_rle_crop_3976577070_0.png temp/1758875208_4182383_1386035593_caba9dda06e35c2242371dc55b10b8ba_rle_crop_3976577097_0.png temp/1758875208_4182383_1386035560_303933f4322dd0b7e0004f49578b2c98_rle_crop_3976577115_0.png temp/1758875208_4182383_1386035560_303933f4322dd0b7e0004f49578b2c98_rle_crop_3976577122_0.png temp/1758875208_4182383_1386038142_1fed7805efab59b4f92b5fa032d205f2_rle_crop_3976576968_0.png temp/1758875208_4182383_1386038140_94b6bdf6a266ed505777356c952d9cf8_rle_crop_3976576974_0.png temp/1758875208_4182383_1386038140_94b6bdf6a266ed505777356c952d9cf8_rle_crop_3976576976_0.png temp/1758875208_4182383_1386038140_94b6bdf6a266ed505777356c952d9cf8_rle_crop_3976576982_0.png temp/1758875208_4182383_1386038140_94b6bdf6a266ed505777356c952d9cf8_rle_crop_3976576983_0.png temp/1758875208_4182383_1386038138_6c6e3b9e5d761766a6b6c3891093cce6_rle_crop_3976576990_0.png temp/1758875208_4182383_1386038138_6c6e3b9e5d761766a6b6c3891093cce6_rle_crop_3976576991_0.png temp/1758875208_4182383_1386038138_6c6e3b9e5d761766a6b6c3891093cce6_rle_crop_3976576993_0.png temp/1758875208_4182383_1386038136_cce810f4277a6a6837e6a10f60282eff_rle_crop_3976577007_0.png temp/1758875208_4182383_1386038136_cce810f4277a6a6837e6a10f60282eff_rle_crop_3976577011_0.png temp/1758875208_4182383_1386037955_f5e351c86db4be7f645acce8e98328fc_rle_crop_3976577015_0.png temp/1758875208_4182383_1386037955_f5e351c86db4be7f645acce8e98328fc_rle_crop_3976577020_0.png temp/1758875208_4182383_1386037914_61f14c37b0115a27ad5222cff0059f6a_rle_crop_3976577036_0.png temp/1758875208_4182383_1386037894_5087993376f1d446f9f898c01dfb9319_rle_crop_3976577046_0.png temp/1758875208_4182383_1386037894_5087993376f1d446f9f898c01dfb9319_rle_crop_3976577047_0.png temp/1758875208_4182383_1386037894_5087993376f1d446f9f898c01dfb9319_rle_crop_3976577051_0.png temp/1758875208_4182383_1386037894_5087993376f1d446f9f898c01dfb9319_rle_crop_3976577052_0.png temp/1758875208_4182383_1386037858_88a1c550e29cca5c5d9af082685efac3_rle_crop_3976577057_0.png temp/1758875208_4182383_1386037850_72d5fd97d6245c749bde7c9e20ee2578_rle_crop_3976577065_0.png temp/1758875208_4182383_1386037850_72d5fd97d6245c749bde7c9e20ee2578_rle_crop_3976577066_0.png temp/1758875208_4182383_1386037850_72d5fd97d6245c749bde7c9e20ee2578_rle_crop_3976577071_0.png temp/1758875208_4182383_1386037850_72d5fd97d6245c749bde7c9e20ee2578_rle_crop_3976577072_0.png temp/1758875208_4182383_1386035648_f2fa7e98faf6e077d8069ab08845feae_rle_crop_3976577078_0.png temp/1758875208_4182383_1386035648_f2fa7e98faf6e077d8069ab08845feae_rle_crop_3976577084_0.png temp/1758875208_4182383_1386035593_caba9dda06e35c2242371dc55b10b8ba_rle_crop_3976577094_0.png temp/1758875208_4182383_1386035593_caba9dda06e35c2242371dc55b10b8ba_rle_crop_3976577102_0.png temp/1758875208_4182383_1386035560_303933f4322dd0b7e0004f49578b2c98_rle_crop_3976577106_0.png temp/1758875208_4182383_1386035560_303933f4322dd0b7e0004f49578b2c98_rle_crop_3976577112_0.png temp/1758875208_4182383_1386035560_303933f4322dd0b7e0004f49578b2c98_rle_crop_3976577113_0.png temp/1758875208_4182383_1386035560_303933f4322dd0b7e0004f49578b2c98_rle_crop_3976577114_0.png temp/1758875208_4182383_1386035526_991d4751f43b97693d1210809b83a299_rle_crop_3976577127_0.png temp/1758875208_4182383_1386038142_1fed7805efab59b4f92b5fa032d205f2_rle_crop_3976576962_0.png temp/1758875208_4182383_1386038142_1fed7805efab59b4f92b5fa032d205f2_rle_crop_3976576963_0.png temp/1758875208_4182383_1386038140_94b6bdf6a266ed505777356c952d9cf8_rle_crop_3976576972_0.png temp/1758875208_4182383_1386038140_94b6bdf6a266ed505777356c952d9cf8_rle_crop_3976576980_0.png temp/1758875208_4182383_1386038138_6c6e3b9e5d761766a6b6c3891093cce6_rle_crop_3976576987_0.png temp/1758875208_4182383_1386038137_b49dac4e1b4372f5fb7b47eb1b864ba2_rle_crop_3976576996_0.png temp/1758875208_4182383_1386038137_b49dac4e1b4372f5fb7b47eb1b864ba2_rle_crop_3976576998_0.png 111it [00:00, 1093.25it/s] 122it [00:00, 1109.61it/s]2025-09-26 10:33:53.166141: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 10:33:53.178251: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 temp/1758875208_4182383_1386038136_cce810f4277a6a6837e6a10f60282eff_rle_crop_3976577002_0.png temp/1758875208_4182383_1386038136_cce810f4277a6a6837e6a10f60282eff_rle_crop_3976577004_0.png temp/1758875208_4182383_1386038136_cce810f4277a6a6837e6a10f60282eff_rle_crop_3976577005_0.png temp/1758875208_4182383_1386037955_f5e351c86db4be7f645acce8e98328fc_rle_crop_3976577019_0.png temp/1758875208_4182383_1386037914_61f14c37b0115a27ad5222cff0059f6a_rle_crop_3976577033_0.png temp/1758875208_4182383_1386037894_5087993376f1d446f9f898c01dfb9319_rle_crop_3976577042_0.png temp/1758875208_4182383_1386037894_5087993376f1d446f9f898c01dfb9319_rle_crop_3976577043_0.png temp/1758875208_4182383_1386037894_5087993376f1d446f9f898c01dfb9319_rle_crop_3976577045_0.png temp/1758875208_4182383_1386037858_88a1c550e29cca5c5d9af082685efac3_rle_crop_3976577058_0.png temp/1758875208_4182383_1386035648_f2fa7e98faf6e077d8069ab08845feae_rle_crop_3976577074_0.png temp/1758875208_4182383_1386035648_f2fa7e98faf6e077d8069ab08845feae_rle_crop_3976577075_0.png temp/1758875208_4182383_1386035648_f2fa7e98faf6e077d8069ab08845feae_rle_crop_3976577077_0.png temp/1758875208_4182383_1386035593_caba9dda06e35c2242371dc55b10b8ba_rle_crop_3976577091_0.png temp/1758875208_4182383_1386035593_caba9dda06e35c2242371dc55b10b8ba_rle_crop_3976577095_0.png temp/1758875208_4182383_1386035560_303933f4322dd0b7e0004f49578b2c98_rle_crop_3976577103_0.png temp/1758875208_4182383_1386035560_303933f4322dd0b7e0004f49578b2c98_rle_crop_3976577104_0.png temp/1758875208_4182383_1386035560_303933f4322dd0b7e0004f49578b2c98_rle_crop_3976577105_0.png temp/1758875208_4182383_1386035560_303933f4322dd0b7e0004f49578b2c98_rle_crop_3976577107_0.png temp/1758875208_4182383_1386035560_303933f4322dd0b7e0004f49578b2c98_rle_crop_3976577108_0.png temp/1758875208_4182383_1386035560_303933f4322dd0b7e0004f49578b2c98_rle_crop_3976577111_0.png temp/1758875208_4182383_1386038140_94b6bdf6a266ed505777356c952d9cf8_rle_crop_3976576977_0.png temp/1758875208_4182383_1386038140_94b6bdf6a266ed505777356c952d9cf8_rle_crop_3976576981_0.png temp/1758875208_4182383_1386038138_6c6e3b9e5d761766a6b6c3891093cce6_rle_crop_3976576992_0.png temp/1758875208_4182383_1386037955_f5e351c86db4be7f645acce8e98328fc_rle_crop_3976577024_0.png temp/1758875208_4182383_1386037914_61f14c37b0115a27ad5222cff0059f6a_rle_crop_3976577041_0.png temp/1758875208_4182383_1386037894_5087993376f1d446f9f898c01dfb9319_rle_crop_3976577055_0.png temp/1758875208_4182383_1386037894_5087993376f1d446f9f898c01dfb9319_rle_crop_3976577056_0.png temp/1758875208_4182383_1386037858_88a1c550e29cca5c5d9af082685efac3_rle_crop_3976577062_0.png temp/1758875208_4182383_1386037858_88a1c550e29cca5c5d9af082685efac3_rle_crop_3976577063_0.png temp/1758875208_4182383_1386037850_72d5fd97d6245c749bde7c9e20ee2578_rle_crop_3976577067_0.png temp/1758875208_4182383_1386035648_f2fa7e98faf6e077d8069ab08845feae_rle_crop_3976577073_0.png temp/1758875208_4182383_1386035593_caba9dda06e35c2242371dc55b10b8ba_rle_crop_3976577090_0.png temp/1758875208_4182383_1386035560_303933f4322dd0b7e0004f49578b2c98_rle_crop_3976577109_0.png temp/1758875208_4182383_1386035560_303933f4322dd0b7e0004f49578b2c98_rle_crop_3976577118_0.png temp/1758875208_4182383_1386035593_caba9dda06e35c2242371dc55b10b8ba_rle_crop_3976577100_0.png Found 122 images belonging to 1 classes. begin to do the prediction : time used to do the prediction : 6.237451076507568 (122,) (122, 10) (122, 1792) shape of features : (122, 1792) shape of new features : (1, 122, 1792) save descriptor for thcl : 3501 time to traite the descriptors : 0.6232340335845947 storage_type for insertDescriptorsMulti : 3 Missing photo l117 : -3976576964 Missing photo l117 : -3976576966 Missing photo l117 : -3976576979 Missing photo l117 : -3976576997 Missing photo l117 : -3976577009 Missing photo l117 : -3976577016 Missing photo l117 : -3976577028 Missing photo l117 : -3976577032 Missing photo l117 : -3976577050 Missing photo l117 : -3976577068 Missing photo l117 : -3976577079 Missing photo l117 : -3976577088 Missing photo l117 : -3976577089 Missing photo l117 : -3976577092 Missing photo l117 : -3976577098 Missing photo l117 : -3976576967 Missing photo l117 : -3976576970 Missing photo l117 : -3976576978 Missing photo l117 : -3976576985 Missing photo l117 : -3976576989 Missing photo l117 : -3976577001 Missing photo l117 : -3976577003 Missing photo l117 : -3976577010 Missing photo l117 : -3976577014 Missing photo l117 : -3976577022 Missing photo l117 : -3976577030 Missing photo l117 : -3976577034 Missing photo l117 : -3976577039 Missing photo l117 : -3976577044 Missing photo l117 : -3976577049 Missing photo l117 : -3976577053 Missing photo l117 : -3976577054 Missing photo l117 : -3976577060 Missing photo l117 : -3976577076 Missing photo l117 : -3976577110 Missing photo l117 : -3976577119 Missing photo l117 : -3976577128 Missing photo l117 : -3976576988 Missing photo l117 : -3976577006 Missing photo l117 : -3976577008 Missing photo l117 : -3976577013 Missing photo l117 : -3976577021 Missing photo l117 : -3976577031 Missing photo l117 : -3976577037 Missing photo l117 : -3976577069 Missing photo l117 : -3976577070 Missing photo l117 : -3976577097 Missing photo l117 : -3976577115 Missing photo l117 : -3976577122 Missing photo l117 : -3976576968 Missing photo l117 : -3976576974 Missing photo l117 : -3976576976 Missing photo l117 : -3976576982 Missing photo l117 : -3976576983 Missing photo l117 : -3976576990 Missing photo l117 : -3976576991 Missing photo l117 : -3976576993 Missing photo l117 : -3976577007 Missing photo l117 : -3976577011 Missing photo l117 : -3976577015 Missing photo l117 : -3976577020 Missing photo l117 : -3976577036 Missing photo l117 : -3976577046 Missing photo l117 : -3976577047 Missing photo l117 : -3976577051 Missing photo l117 : -3976577052 Missing photo l117 : -3976577057 Missing photo l117 : -3976577065 Missing photo l117 : -3976577066 Missing photo l117 : -3976577071 Missing photo l117 : -3976577072 Missing photo l117 : -3976577078 Missing photo l117 : -3976577084 Missing photo l117 : -3976577094 Missing photo l117 : -3976577102 Missing photo l117 : -3976577106 Missing photo l117 : -3976577112 Missing photo l117 : -3976577113 Missing photo l117 : -3976577114 Missing photo l117 : -3976577127 Missing photo l117 : -3976576962 Missing photo l117 : -3976576963 Missing photo l117 : -3976576972 Missing photo l117 : -3976576980 Missing photo l117 : -3976576987 Missing photo l117 : -3976576996 Missing photo l117 : -3976576998 Missing photo l117 : -3976577002 Missing photo l117 : -3976577004 Missing photo l117 : -3976577005 Missing photo l117 : -3976577019 Missing photo l117 : -3976577033 Missing photo l117 : -3976577042 Missing photo l117 : -3976577043 Missing photo l117 : -3976577045 Missing photo l117 : -3976577058 Missing photo l117 : -3976577074 Missing photo l117 : -3976577075 Missing photo l117 : -3976577077 Missing photo l117 : -3976577091 Missing photo l117 : -3976577095 Missing photo l117 : -3976577103 Missing photo l117 : -3976577104 Missing photo l117 : -3976577105 Missing photo l117 : -3976577107 Missing photo l117 : -3976577108 Missing photo l117 : -3976577111 Missing photo l117 : -3976576977 Missing photo l117 : -3976576981 Missing photo l117 : -3976576992 Missing photo l117 : -3976577024 Missing photo l117 : -3976577041 Missing photo l117 : -3976577055 Missing photo l117 : -3976577056 Missing photo l117 : -3976577062 Missing photo l117 : -3976577063 Missing photo l117 : -3976577067 Missing photo l117 : -3976577073 Missing photo l117 : -3976577090 Missing photo l117 : -3976577109 Missing photo l117 : -3976577118 Missing photo l117 : -3976577100 To insert : -3976576964 Missing photo l134 : -3976576964 To insert : -3976576966 Missing photo l134 : -3976576966 To insert : -3976576979 Missing photo l134 : -3976576979 To insert : -3976576997 Missing photo l134 : -3976576997 To insert : -3976577009 Missing photo l134 : -3976577009 To insert : -3976577016 Missing photo l134 : -3976577016 To insert : -3976577028 Missing photo l134 : -3976577028 To insert : -3976577032 Missing photo l134 : -3976577032 To insert : -3976577050 Missing photo l134 : -3976577050 To insert : -3976577068 Missing photo l134 : -3976577068 To insert : -3976577079 Missing photo l134 : -3976577079 To insert : -3976577088 Missing photo l134 : -3976577088 To insert : -3976577089 Missing photo l134 : -3976577089 To insert : -3976577092 Missing photo l134 : -3976577092 To insert : -3976577098 Missing photo l134 : -3976577098 To insert : -3976576967 Missing photo l134 : -3976576967 To insert : -3976576970 Missing photo l134 : -3976576970 To insert : -3976576978 Missing photo l134 : -3976576978 To insert : -3976576985 Missing photo l134 : -3976576985 To insert : -3976576989 Missing photo l134 : -3976576989 To insert : -3976577001 Missing photo l134 : -3976577001 To insert : -3976577003 Missing photo l134 : -3976577003 To insert : -3976577010 Missing photo l134 : -3976577010 To insert : -3976577014 Missing photo l134 : -3976577014 To insert : -3976577022 Missing photo l134 : -3976577022 To insert : -3976577030 Missing photo l134 : -3976577030 To insert : -3976577034 Missing photo l134 : -3976577034 To insert : -3976577039 Missing photo l134 : -3976577039 To insert : -3976577044 Missing photo l134 : -3976577044 To insert : -3976577049 Missing photo l134 : -3976577049 To insert : -3976577053 Missing photo l134 : -3976577053 To insert : -3976577054 Missing photo l134 : -3976577054 To insert : -3976577060 Missing photo l134 : -3976577060 To insert : -3976577076 Missing photo l134 : -3976577076 To insert : -3976577110 Missing photo l134 : -3976577110 To insert : -3976577119 Missing photo l134 : -3976577119 To insert : -3976577128 Missing photo l134 : -3976577128 To insert : -3976576988 Missing photo l134 : -3976576988 To insert : -3976577006 Missing photo l134 : -3976577006 To insert : -3976577008 Missing photo l134 : -3976577008 To insert : -3976577013 Missing photo l134 : -3976577013 To insert : -3976577021 Missing photo l134 : -3976577021 To insert : -3976577031 Missing photo l134 : -3976577031 To insert : -3976577037 Missing photo l134 : -3976577037 To insert : -3976577069 Missing photo l134 : -3976577069 To insert : -3976577070 Missing photo l134 : -3976577070 To insert : -3976577097 Missing photo l134 : -3976577097 To insert : -3976577115 Missing photo l134 : -3976577115 To insert : -3976577122 Missing photo l134 : -3976577122 To insert : -3976576968 Missing photo l134 : -3976576968 To insert : -3976576974 Missing photo l134 : -3976576974 To insert : -3976576976 Missing photo l134 : -3976576976 To insert : -3976576982 Missing photo l134 : -3976576982 To insert : -3976576983 Missing photo l134 : -3976576983 To insert : -3976576990 Missing photo l134 : -3976576990 To insert : -3976576991 Missing photo l134 : -3976576991 To insert : -3976576993 Missing photo l134 : -3976576993 To insert : -3976577007 Missing photo l134 : -3976577007 To insert : -3976577011 Missing photo l134 : -3976577011 To insert : -3976577015 Missing photo l134 : -3976577015 To insert : -3976577020 Missing photo l134 : -3976577020 To insert : -3976577036 Missing photo l134 : -3976577036 To insert : -3976577046 Missing photo l134 : -3976577046 To insert : -3976577047 Missing photo l134 : -3976577047 To insert : -3976577051 Missing photo l134 : -3976577051 To insert : -3976577052 Missing photo l134 : -3976577052 To insert : -3976577057 Missing photo l134 : -3976577057 To insert : -3976577065 Missing photo l134 : -3976577065 To insert : -3976577066 Missing photo l134 : -3976577066 To insert : -3976577071 Missing photo l134 : -3976577071 To insert : -3976577072 Missing photo l134 : -3976577072 To insert : -3976577078 Missing photo l134 : -3976577078 To insert : -3976577084 Missing photo l134 : -3976577084 To insert : -3976577094 Missing photo l134 : -3976577094 To insert : -3976577102 Missing photo l134 : -3976577102 To insert : -3976577106 Missing photo l134 : -3976577106 To insert : -3976577112 Missing photo l134 : -3976577112 To insert : -3976577113 Missing photo l134 : -3976577113 To insert : -3976577114 Missing photo l134 : -3976577114 To insert : -3976577127 Missing photo l134 : -3976577127 To insert : -3976576962 Missing photo l134 : -3976576962 To insert : -3976576963 Missing photo l134 : -3976576963 To insert : -3976576972 Missing photo l134 : -3976576972 To insert : -3976576980 Missing photo l134 : -3976576980 To insert : -3976576987 Missing photo l134 : -3976576987 To insert : -3976576996 Missing photo l134 : -3976576996 To insert : -3976576998 Missing photo l134 : -3976576998 To insert : -3976577002 Missing photo l134 : -3976577002 To insert : -3976577004 Missing photo l134 : -3976577004 To insert : -3976577005 Missing photo l134 : -3976577005 To insert : -3976577019 Missing photo l134 : -3976577019 To insert : -3976577033 Missing photo l134 : -3976577033 To insert : -3976577042 Missing photo l134 : -3976577042 To insert : -3976577043 Missing photo l134 : -3976577043 To insert : -3976577045 Missing photo l134 : -3976577045 To insert : -3976577058 Missing photo l134 : -3976577058 To insert : -3976577074 Missing photo l134 : -3976577074 To insert : -3976577075 Missing photo l134 : -3976577075 To insert : -3976577077 Missing photo l134 : -3976577077 To insert : -3976577091 Missing photo l134 : -3976577091 To insert : -3976577095 Missing photo l134 : -3976577095 To insert : -3976577103 Missing photo l134 : -3976577103 To insert : -3976577104 Missing photo l134 : -3976577104 To insert : -3976577105 Missing photo l134 : -3976577105 To insert : -3976577107 Missing photo l134 : -3976577107 To insert : -3976577108 Missing photo l134 : -3976577108 To insert : -3976577111 Missing photo l134 : -3976577111 To insert : -3976576977 Missing photo l134 : -3976576977 To insert : -3976576981 Missing photo l134 : -3976576981 To insert : -3976576992 Missing photo l134 : -3976576992 To insert : -3976577024 Missing photo l134 : -3976577024 To insert : -3976577041 Missing photo l134 : -3976577041 To insert : -3976577055 Missing photo l134 : -3976577055 To insert : -3976577056 Missing photo l134 : -3976577056 To insert : -3976577062 Missing photo l134 : -3976577062 To insert : -3976577063 Missing photo l134 : -3976577063 To insert : -3976577067 Missing photo l134 : -3976577067 To insert : -3976577073 Missing photo l134 : -3976577073 To insert : -3976577090 Missing photo l134 : -3976577090 To insert : -3976577109 Missing photo l134 : -3976577109 To insert : -3976577118 Missing photo l134 : -3976577118 To insert : -3976577100 Missing photo l134 : -3976577100 time to insert the descriptors : 26.68494176864624 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : tfhub_classification2 we use saveGeneral [1386038142, 1386038140, 1386038138, 1386038137, 1386038136, 1386037955, 1386037926, 1386037914, 1386037894, 1386037858, 1386037850, 1386035648, 1386035593, 1386035560, 1386035526] Looping around the photos to save general results len do output : 122 /-3976576964Didn't retrieve data . /-3976576966Didn't retrieve data . /-3976576979Didn't retrieve data . /-3976576997Didn't retrieve data . /-3976577009Didn't retrieve data . /-3976577016Didn't retrieve data . /-3976577028Didn't retrieve data . /-3976577032Didn't retrieve data . /-3976577050Didn't retrieve data . /-3976577068Didn't retrieve data . /-3976577079Didn't retrieve data . /-3976577088Didn't retrieve data . /-3976577089Didn't retrieve data . /-3976577092Didn't retrieve data . /-3976577098Didn't retrieve data . /-3976576967Didn't retrieve data . /-3976576970Didn't retrieve data . /-3976576978Didn't retrieve data . /-3976576985Didn't retrieve data . /-3976576989Didn't retrieve data . /-3976577001Didn't retrieve data . /-3976577003Didn't retrieve data . /-3976577010Didn't retrieve data . /-3976577014Didn't retrieve data . /-3976577022Didn't retrieve data . /-3976577030Didn't retrieve data . /-3976577034Didn't retrieve data . /-3976577039Didn't retrieve data . /-3976577044Didn't retrieve data . /-3976577049Didn't retrieve data . /-3976577053Didn't retrieve data . /-3976577054Didn't retrieve data . /-3976577060Didn't retrieve data . /-3976577076Didn't retrieve data . /-3976577110Didn't retrieve data . /-3976577119Didn't retrieve data . /-3976577128Didn't retrieve data . /-3976576988Didn't retrieve data . /-3976577006Didn't retrieve data . /-3976577008Didn't retrieve data . /-3976577013Didn't retrieve data . /-3976577021Didn't retrieve data . /-3976577031Didn't retrieve data . /-3976577037Didn't retrieve data . /-3976577069Didn't retrieve data . /-3976577070Didn't retrieve data . /-3976577097Didn't retrieve data . /-3976577115Didn't retrieve data . /-3976577122Didn't retrieve data . /-3976576968Didn't retrieve data . /-3976576974Didn't retrieve data . /-3976576976Didn't retrieve data . /-3976576982Didn't retrieve data . /-3976576983Didn't retrieve data . /-3976576990Didn't retrieve data . /-3976576991Didn't retrieve data . /-3976576993Didn't retrieve data . /-3976577007Didn't retrieve data . /-3976577011Didn't retrieve data . /-3976577015Didn't retrieve data . /-3976577020Didn't retrieve data . /-3976577036Didn't retrieve data . /-3976577046Didn't retrieve data . /-3976577047Didn't retrieve data . /-3976577051Didn't retrieve data . /-3976577052Didn't retrieve data . /-3976577057Didn't retrieve data . /-3976577065Didn't retrieve data . /-3976577066Didn't retrieve data . /-3976577071Didn't retrieve data . /-3976577072Didn't retrieve data . /-3976577078Didn't retrieve data . /-3976577084Didn't retrieve data . /-3976577094Didn't retrieve data . /-3976577102Didn't retrieve data . /-3976577106Didn't retrieve data . /-3976577112Didn't retrieve data . /-3976577113Didn't retrieve data . /-3976577114Didn't retrieve data . /-3976577127Didn't retrieve data . /-3976576962Didn't retrieve data . /-3976576963Didn't retrieve data . /-3976576972Didn't retrieve data . /-3976576980Didn't retrieve data . /-3976576987Didn't retrieve data . /-3976576996Didn't retrieve data . /-3976576998Didn't retrieve data . /-3976577002Didn't retrieve data . /-3976577004Didn't retrieve data . /-3976577005Didn't retrieve data . /-3976577019Didn't retrieve data . /-3976577033Didn't retrieve data . /-3976577042Didn't retrieve data . /-3976577043Didn't retrieve data . /-3976577045Didn't retrieve data . /-3976577058Didn't retrieve data . /-3976577074Didn't retrieve data . /-3976577075Didn't retrieve data . /-3976577077Didn't retrieve data . /-3976577091Didn't retrieve data . /-3976577095Didn't retrieve data . /-3976577103Didn't retrieve data . /-3976577104Didn't retrieve data . /-3976577105Didn't retrieve data . /-3976577107Didn't retrieve data . /-3976577108Didn't retrieve data . /-3976577111Didn't retrieve data . /-3976576977Didn't retrieve data . /-3976576981Didn't retrieve data . /-3976576992Didn't retrieve data . /-3976577024Didn't retrieve data . /-3976577041Didn't retrieve data . /-3976577055Didn't retrieve data . /-3976577056Didn't retrieve data . /-3976577062Didn't retrieve data . /-3976577063Didn't retrieve data . /-3976577067Didn't retrieve data . /-3976577073Didn't retrieve data . /-3976577090Didn't retrieve data . /-3976577109Didn't retrieve data . /-3976577118Didn't retrieve data . /-3976577100Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038142', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038140', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038138', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038137', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038136', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037955', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037926', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037914', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037894', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037858', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037850', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035648', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035593', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035560', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035526', None, None, None, None, None, '3784202') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 137 time used for this insertion : 0.030579090118408203 save_final save missing photos in datou_result : time spend for datou_step_exec : 58.30854940414429 time spend to save output : 0.0335841178894043 total time spend for step 6 : 58.34213352203369 step7:merge_mask_thcl_custom Fri Sep 26 10:34:26 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 167 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1386038142, 1386038140, 1386038138, 1386038137, 1386038136, 1386037955, 1386037926, 1386037914, 1386037894, 1386037858, 1386037850, 1386035648, 1386035593, 1386035560, 1386035526] Looping around the photos to save general results len do output : 15 /1386038142Didn't retrieve data .Didn't retrieve data . /1386038140Didn't retrieve data .Didn't retrieve data . /1386038138Didn't retrieve data .Didn't retrieve data . /1386038137Didn't retrieve data .Didn't retrieve data . /1386038136Didn't retrieve data .Didn't retrieve data . /1386037955Didn't retrieve data .Didn't retrieve data . /1386037926Didn't retrieve data .Didn't retrieve data . /1386037914Didn't retrieve data .Didn't retrieve data . /1386037894Didn't retrieve data .Didn't retrieve data . /1386037858Didn't retrieve data .Didn't retrieve data . /1386037850Didn't retrieve data .Didn't retrieve data . /1386035648Didn't retrieve data .Didn't retrieve data . /1386035593Didn't retrieve data .Didn't retrieve data . /1386035560Didn't retrieve data .Didn't retrieve data . /1386035526Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038142', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038140', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038138', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038137', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038136', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037955', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037926', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037914', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037894', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037858', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037850', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035648', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035593', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035560', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035526', None, None, None, None, None, '3784202') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 45 time used for this insertion : 0.013436079025268555 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.06945013999938965 time spend to save output : 0.014114618301391602 total time spend for step 7 : 0.08356475830078125 step8:rle_unique_nms_with_priority Fri Sep 26 10:34:26 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 9 nb_hashtags : 4 time to prepare the origin masks : 9.17041563987732 time for calcul the mask position with numpy : 0.545665979385376 nb_pixel_total : 10137697 time to create 1 rle with new method : 1.0448229312896729 time for calcul the mask position with numpy : 0.06377005577087402 nb_pixel_total : 87202 time to create 1 rle with old method : 0.0978386402130127 time for calcul the mask position with numpy : 0.06293082237243652 nb_pixel_total : 207245 time to create 1 rle with new method : 0.8871486186981201 time for calcul the mask position with numpy : 0.06955790519714355 nb_pixel_total : 1120726 time to create 1 rle with new method : 0.6782486438751221 time for calcul the mask position with numpy : 0.0379793643951416 nb_pixel_total : 26663 time to create 1 rle with old method : 0.029857873916625977 time for calcul the mask position with numpy : 0.04081416130065918 nb_pixel_total : 360355 time to create 1 rle with new method : 1.1835048198699951 time for calcul the mask position with numpy : 0.07224535942077637 nb_pixel_total : 298402 time to create 1 rle with new method : 1.0377233028411865 time for calcul the mask position with numpy : 0.05789470672607422 nb_pixel_total : 15757 time to create 1 rle with old method : 0.016946077346801758 time for calcul the mask position with numpy : 0.058522939682006836 nb_pixel_total : 65855 time to create 1 rle with old method : 0.07060122489929199 time for calcul the mask position with numpy : 0.05727338790893555 nb_pixel_total : 10338 time to create 1 rle with old method : 0.010973930358886719 create new chi : 6.374929904937744 time to delete rle : 0.010672569274902344 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 14871 TO DO : save crop sub photo not yet done ! save time : 0.7697432041168213 nb_obj : 14 nb_hashtags : 6 time to prepare the origin masks : 16.65975522994995 time for calcul the mask position with numpy : 0.5777745246887207 nb_pixel_total : 9096493 time to create 1 rle with new method : 1.259911060333252 time for calcul the mask position with numpy : 0.03668355941772461 nb_pixel_total : 18312 time to create 1 rle with old method : 0.02009439468383789 time for calcul the mask position with numpy : 0.03971266746520996 nb_pixel_total : 41917 time to create 1 rle with old method : 0.04611659049987793 time for calcul the mask position with numpy : 0.0421595573425293 nb_pixel_total : 1014079 time to create 1 rle with new method : 0.9250655174255371 time for calcul the mask position with numpy : 0.036551713943481445 nb_pixel_total : 57397 time to create 1 rle with old method : 0.06400203704833984 time for calcul the mask position with numpy : 0.043378353118896484 nb_pixel_total : 13029 time to create 1 rle with old method : 0.016774415969848633 time for calcul the mask position with numpy : 0.03861665725708008 nb_pixel_total : 12597 time to create 1 rle with old method : 0.014518976211547852 time for calcul the mask position with numpy : 0.036855459213256836 nb_pixel_total : 50198 time to create 1 rle with old method : 0.0550687313079834 time for calcul the mask position with numpy : 0.039099693298339844 nb_pixel_total : 374984 time to create 1 rle with new method : 0.8990256786346436 time for calcul the mask position with numpy : 0.03652811050415039 nb_pixel_total : 155220 time to create 1 rle with new method : 0.739067792892456 time for calcul the mask position with numpy : 0.03986310958862305 nb_pixel_total : 115289 time to create 1 rle with old method : 0.13022947311401367 time for calcul the mask position with numpy : 0.03679323196411133 nb_pixel_total : 9268 time to create 1 rle with old method : 0.010584592819213867 time for calcul the mask position with numpy : 0.10497832298278809 nb_pixel_total : 312656 time to create 1 rle with new method : 0.8279976844787598 time for calcul the mask position with numpy : 0.05359220504760742 nb_pixel_total : 1052116 time to create 1 rle with new method : 0.9510881900787354 time for calcul the mask position with numpy : 0.03929948806762695 nb_pixel_total : 6685 time to create 1 rle with old method : 0.007452249526977539 create new chi : 7.441503047943115 time to delete rle : 0.0024983882904052734 batch 1 Loaded 15 chid ids of type : 3726 Number RLEs to save : 18719 TO DO : save crop sub photo not yet done ! save time : 1.1456193923950195 nb_obj : 9 nb_hashtags : 4 time to prepare the origin masks : 10.640600204467773 time for calcul the mask position with numpy : 0.9200453758239746 nb_pixel_total : 9390579 time to create 1 rle with new method : 1.2888524532318115 time for calcul the mask position with numpy : 0.06283736228942871 nb_pixel_total : 200695 time to create 1 rle with new method : 1.1427226066589355 time for calcul the mask position with numpy : 0.07503557205200195 nb_pixel_total : 1107631 time to create 1 rle with new method : 0.7817435264587402 time for calcul the mask position with numpy : 0.0608060359954834 nb_pixel_total : 88393 time to create 1 rle with old method : 0.09673166275024414 time for calcul the mask position with numpy : 0.09035301208496094 nb_pixel_total : 139507 time to create 1 rle with old method : 0.1755080223083496 time for calcul the mask position with numpy : 0.06038665771484375 nb_pixel_total : 26011 time to create 1 rle with old method : 0.03116917610168457 time for calcul the mask position with numpy : 0.08053374290466309 nb_pixel_total : 94859 time to create 1 rle with old method : 0.10814762115478516 time for calcul the mask position with numpy : 0.06384420394897461 nb_pixel_total : 48826 time to create 1 rle with old method : 0.057727813720703125 time for calcul the mask position with numpy : 0.08452558517456055 nb_pixel_total : 293351 time to create 1 rle with new method : 0.7231802940368652 time for calcul the mask position with numpy : 0.0670325756072998 nb_pixel_total : 940388 time to create 1 rle with new method : 0.8546035289764404 create new chi : 7.065451622009277 time to delete rle : 0.002381563186645508 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 16405 TO DO : save crop sub photo not yet done ! save time : 0.9587094783782959 nb_obj : 6 nb_hashtags : 3 time to prepare the origin masks : 7.446175813674927 time for calcul the mask position with numpy : 0.5416748523712158 nb_pixel_total : 11934584 time to create 1 rle with new method : 0.58201003074646 time for calcul the mask position with numpy : 0.03814888000488281 nb_pixel_total : 5863 time to create 1 rle with old method : 0.0067212581634521484 time for calcul the mask position with numpy : 0.03509163856506348 nb_pixel_total : 17740 time to create 1 rle with old method : 0.01999688148498535 time for calcul the mask position with numpy : 0.10412335395812988 nb_pixel_total : 269073 time to create 1 rle with new method : 0.7216746807098389 time for calcul the mask position with numpy : 0.03879499435424805 nb_pixel_total : 10713 time to create 1 rle with old method : 0.017268896102905273 time for calcul the mask position with numpy : 0.058853864669799805 nb_pixel_total : 66466 time to create 1 rle with old method : 0.20911407470703125 time for calcul the mask position with numpy : 0.03698420524597168 nb_pixel_total : 25801 time to create 1 rle with old method : 0.02926039695739746 create new chi : 2.5408201217651367 time to delete rle : 0.001058340072631836 batch 1 Loaded 7 chid ids of type : 3726 Number RLEs to save : 7436 TO DO : save crop sub photo not yet done ! save time : 0.4438204765319824 nb_obj : 12 nb_hashtags : 5 time to prepare the origin masks : 13.560116291046143 time for calcul the mask position with numpy : 0.5674054622650146 nb_pixel_total : 10546033 time to create 1 rle with new method : 1.4869649410247803 time for calcul the mask position with numpy : 0.07966256141662598 nb_pixel_total : 211474 time to create 1 rle with new method : 0.756554126739502 time for calcul the mask position with numpy : 0.062310218811035156 nb_pixel_total : 178798 time to create 1 rle with new method : 0.7773933410644531 time for calcul the mask position with numpy : 0.03755331039428711 nb_pixel_total : 45912 time to create 1 rle with old method : 0.05192136764526367 time for calcul the mask position with numpy : 0.0411074161529541 nb_pixel_total : 7691 time to create 1 rle with old method : 0.01050567626953125 time for calcul the mask position with numpy : 0.05417966842651367 nb_pixel_total : 27296 time to create 1 rle with old method : 0.031001806259155273 time for calcul the mask position with numpy : 0.22802519798278809 nb_pixel_total : 607373 time to create 1 rle with new method : 0.8564958572387695 time for calcul the mask position with numpy : 0.04001164436340332 nb_pixel_total : 81831 time to create 1 rle with old method : 0.10170483589172363 time for calcul the mask position with numpy : 0.03825044631958008 nb_pixel_total : 206983 time to create 1 rle with new method : 1.0435585975646973 time for calcul the mask position with numpy : 0.06519341468811035 nb_pixel_total : 6236 time to create 1 rle with old method : 0.009823083877563477 time for calcul the mask position with numpy : 0.07899856567382812 nb_pixel_total : 297696 time to create 1 rle with new method : 0.7282295227050781 time for calcul the mask position with numpy : 0.05893111228942871 nb_pixel_total : 100727 time to create 1 rle with old method : 0.11783647537231445 time for calcul the mask position with numpy : 0.06960821151733398 nb_pixel_total : 12190 time to create 1 rle with old method : 0.013173341751098633 create new chi : 7.687858819961548 time to delete rle : 0.0014312267303466797 batch 1 Loaded 13 chid ids of type : 3726 Number RLEs to save : 15009 TO DO : save crop sub photo not yet done ! save time : 0.9396865367889404 nb_obj : 9 nb_hashtags : 3 time to prepare the origin masks : 10.167414903640747 time for calcul the mask position with numpy : 0.3491489887237549 nb_pixel_total : 7984228 time to create 1 rle with new method : 0.7813091278076172 time for calcul the mask position with numpy : 0.0422823429107666 nb_pixel_total : 139094 time to create 1 rle with old method : 0.16072893142700195 time for calcul the mask position with numpy : 0.04053068161010742 nb_pixel_total : 1087 time to create 1 rle with old method : 0.002170562744140625 time for calcul the mask position with numpy : 0.04034686088562012 nb_pixel_total : 147360 time to create 1 rle with old method : 0.17393851280212402 time for calcul the mask position with numpy : 0.08636736869812012 nb_pixel_total : 22324 time to create 1 rle with old method : 0.028435707092285156 time for calcul the mask position with numpy : 0.10606241226196289 nb_pixel_total : 78351 time to create 1 rle with old method : 0.09509682655334473 time for calcul the mask position with numpy : 0.08549714088439941 nb_pixel_total : 2452298 time to create 1 rle with new method : 0.5519447326660156 time for calcul the mask position with numpy : 0.08541202545166016 nb_pixel_total : 1473191 time to create 1 rle with new method : 0.5721759796142578 time for calcul the mask position with numpy : 0.06374192237854004 nb_pixel_total : 315 time to create 1 rle with old method : 0.0006382465362548828 time for calcul the mask position with numpy : 0.06914544105529785 nb_pixel_total : 31992 time to create 1 rle with old method : 0.0452268123626709 create new chi : 3.533050775527954 time to delete rle : 0.003815889358520508 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 16851 TO DO : save crop sub photo not yet done ! save time : 0.9879250526428223 nb_obj : 5 nb_hashtags : 2 time to prepare the origin masks : 9.978589534759521 time for calcul the mask position with numpy : 0.3699958324432373 nb_pixel_total : 8733796 time to create 1 rle with new method : 0.6939971446990967 time for calcul the mask position with numpy : 0.10653495788574219 nb_pixel_total : 23191 time to create 1 rle with old method : 0.05502748489379883 time for calcul the mask position with numpy : 0.04477643966674805 nb_pixel_total : 75469 time to create 1 rle with old method : 0.09939742088317871 time for calcul the mask position with numpy : 0.14919376373291016 nb_pixel_total : 43418 time to create 1 rle with old method : 0.05826306343078613 time for calcul the mask position with numpy : 0.06287217140197754 nb_pixel_total : 1810417 time to create 1 rle with new method : 0.9796500205993652 time for calcul the mask position with numpy : 0.07705187797546387 nb_pixel_total : 1643949 time to create 1 rle with new method : 0.8829398155212402 create new chi : 3.7379348278045654 time to delete rle : 0.0017287731170654297 batch 1 Loaded 6 chid ids of type : 3726 Number RLEs to save : 17194 TO DO : save crop sub photo not yet done ! save time : 1.34548020362854 nb_obj : 9 nb_hashtags : 3 time to prepare the origin masks : 11.638694286346436 time for calcul the mask position with numpy : 0.3523871898651123 nb_pixel_total : 9379799 time to create 1 rle with new method : 0.7589313983917236 time for calcul the mask position with numpy : 0.04292106628417969 nb_pixel_total : 60386 time to create 1 rle with old method : 0.06846380233764648 time for calcul the mask position with numpy : 0.047315120697021484 nb_pixel_total : 1443 time to create 1 rle with old method : 0.002290964126586914 time for calcul the mask position with numpy : 0.05142045021057129 nb_pixel_total : 42488 time to create 1 rle with old method : 0.05428671836853027 time for calcul the mask position with numpy : 0.05915427207946777 nb_pixel_total : 40045 time to create 1 rle with old method : 0.04683041572570801 time for calcul the mask position with numpy : 0.049123525619506836 nb_pixel_total : 271880 time to create 1 rle with new method : 1.110034704208374 time for calcul the mask position with numpy : 0.045476675033569336 nb_pixel_total : 11155 time to create 1 rle with old method : 0.012660980224609375 time for calcul the mask position with numpy : 0.06072235107421875 nb_pixel_total : 830765 time to create 1 rle with new method : 0.8018827438354492 time for calcul the mask position with numpy : 0.0505070686340332 nb_pixel_total : 439512 time to create 1 rle with new method : 0.8942718505859375 time for calcul the mask position with numpy : 0.08655905723571777 nb_pixel_total : 1252767 time to create 1 rle with new method : 0.6543195247650146 create new chi : 5.492694616317749 time to delete rle : 0.0024895668029785156 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 18445 TO DO : save crop sub photo not yet done ! save time : 1.1017203330993652 nb_obj : 15 nb_hashtags : 6 time to prepare the origin masks : 19.99704670906067 time for calcul the mask position with numpy : 0.41806626319885254 nb_pixel_total : 8976180 time to create 1 rle with new method : 0.9737906455993652 time for calcul the mask position with numpy : 0.05635499954223633 nb_pixel_total : 12910 time to create 1 rle with old method : 0.014681339263916016 time for calcul the mask position with numpy : 0.06014132499694824 nb_pixel_total : 41670 time to create 1 rle with old method : 0.05343341827392578 time for calcul the mask position with numpy : 0.048784732818603516 nb_pixel_total : 183163 time to create 1 rle with new method : 0.6382322311401367 time for calcul the mask position with numpy : 0.07153630256652832 nb_pixel_total : 150787 time to create 1 rle with new method : 1.818960189819336 time for calcul the mask position with numpy : 0.06432342529296875 nb_pixel_total : 323071 time to create 1 rle with new method : 0.9076993465423584 time for calcul the mask position with numpy : 0.07942533493041992 nb_pixel_total : 527144 time to create 1 rle with new method : 0.811323881149292 time for calcul the mask position with numpy : 0.06255602836608887 nb_pixel_total : 144301 time to create 1 rle with old method : 0.16835427284240723 time for calcul the mask position with numpy : 0.07176327705383301 nb_pixel_total : 15991 time to create 1 rle with old method : 0.018610000610351562 time for calcul the mask position with numpy : 0.06232857704162598 nb_pixel_total : 130920 time to create 1 rle with old method : 0.1533806324005127 time for calcul the mask position with numpy : 0.06157636642456055 nb_pixel_total : 55773 time to create 1 rle with old method : 0.06429719924926758 time for calcul the mask position with numpy : 0.06031990051269531 nb_pixel_total : 69810 time to create 1 rle with old method : 0.0758056640625 time for calcul the mask position with numpy : 0.06221795082092285 nb_pixel_total : 227291 time to create 1 rle with new method : 0.6304123401641846 time for calcul the mask position with numpy : 0.06979775428771973 nb_pixel_total : 22231 time to create 1 rle with old method : 0.026271343231201172 time for calcul the mask position with numpy : 0.05696725845336914 nb_pixel_total : 449915 time to create 1 rle with new method : 0.5922689437866211 time for calcul the mask position with numpy : 0.04888319969177246 nb_pixel_total : 999083 time to create 1 rle with new method : 0.8397090435028076 create new chi : 9.70977258682251 time to delete rle : 0.003188610076904297 batch 1 Loaded 16 chid ids of type : 3726 Number RLEs to save : 22051 TO DO : save crop sub photo not yet done ! save time : 1.3583989143371582 nb_obj : 7 nb_hashtags : 3 time to prepare the origin masks : 13.212727069854736 time for calcul the mask position with numpy : 0.5431387424468994 nb_pixel_total : 10695775 time to create 1 rle with new method : 0.709064245223999 time for calcul the mask position with numpy : 0.08446621894836426 nb_pixel_total : 31123 time to create 1 rle with old method : 0.036466360092163086 time for calcul the mask position with numpy : 0.06479811668395996 nb_pixel_total : 379770 time to create 1 rle with new method : 0.7029478549957275 time for calcul the mask position with numpy : 0.06923699378967285 nb_pixel_total : 455602 time to create 1 rle with new method : 0.924100399017334 time for calcul the mask position with numpy : 0.04150056838989258 nb_pixel_total : 49179 time to create 1 rle with old method : 0.06048393249511719 time for calcul the mask position with numpy : 0.042903900146484375 nb_pixel_total : 90729 time to create 1 rle with old method : 0.10221385955810547 time for calcul the mask position with numpy : 0.04614853858947754 nb_pixel_total : 6584 time to create 1 rle with old method : 0.03032231330871582 time for calcul the mask position with numpy : 0.09966611862182617 nb_pixel_total : 621478 time to create 1 rle with new method : 0.8174433708190918 create new chi : 4.564054012298584 time to delete rle : 0.0014271736145019531 batch 1 Loaded 8 chid ids of type : 3726 Number RLEs to save : 11810 TO DO : save crop sub photo not yet done ! save time : 0.9721646308898926 nb_obj : 8 nb_hashtags : 5 time to prepare the origin masks : 12.769354820251465 time for calcul the mask position with numpy : 0.5227949619293213 nb_pixel_total : 11327310 time to create 1 rle with new method : 0.5853235721588135 time for calcul the mask position with numpy : 0.07343268394470215 nb_pixel_total : 123498 time to create 1 rle with old method : 0.13898944854736328 time for calcul the mask position with numpy : 0.0841832160949707 nb_pixel_total : 25388 time to create 1 rle with old method : 0.029581308364868164 time for calcul the mask position with numpy : 0.0647883415222168 nb_pixel_total : 337151 time to create 1 rle with new method : 0.5857188701629639 time for calcul the mask position with numpy : 0.06580877304077148 nb_pixel_total : 2193 time to create 1 rle with old method : 0.0033822059631347656 time for calcul the mask position with numpy : 0.06480622291564941 nb_pixel_total : 2622 time to create 1 rle with old method : 0.0037267208099365234 time for calcul the mask position with numpy : 0.0609128475189209 nb_pixel_total : 215320 time to create 1 rle with new method : 0.5769829750061035 time for calcul the mask position with numpy : 0.0410616397857666 nb_pixel_total : 198839 time to create 1 rle with new method : 0.6811051368713379 time for calcul the mask position with numpy : 0.05931591987609863 nb_pixel_total : 97919 time to create 1 rle with old method : 0.1130533218383789 create new chi : 3.943448543548584 time to delete rle : 0.0013151168823242188 batch 1 Loaded 9 chid ids of type : 3726 Number RLEs to save : 9164 TO DO : save crop sub photo not yet done ! save time : 0.5346078872680664 nb_obj : 11 nb_hashtags : 4 time to prepare the origin masks : 12.622974395751953 time for calcul the mask position with numpy : 0.3164658546447754 nb_pixel_total : 7880327 time to create 1 rle with new method : 1.6827723979949951 time for calcul the mask position with numpy : 0.0561981201171875 nb_pixel_total : 320132 time to create 1 rle with new method : 1.029536485671997 time for calcul the mask position with numpy : 0.04666590690612793 nb_pixel_total : 754791 time to create 1 rle with new method : 0.8461358547210693 time for calcul the mask position with numpy : 0.04484391212463379 nb_pixel_total : 22406 time to create 1 rle with old method : 0.024826765060424805 time for calcul the mask position with numpy : 0.0567631721496582 nb_pixel_total : 1630292 time to create 1 rle with new method : 0.9207868576049805 time for calcul the mask position with numpy : 0.03910112380981445 nb_pixel_total : 315360 time to create 1 rle with new method : 0.6558411121368408 time for calcul the mask position with numpy : 0.045331478118896484 nb_pixel_total : 473697 time to create 1 rle with new method : 0.6009600162506104 time for calcul the mask position with numpy : 0.04749941825866699 nb_pixel_total : 164217 time to create 1 rle with new method : 0.5829224586486816 time for calcul the mask position with numpy : 0.06634902954101562 nb_pixel_total : 131037 time to create 1 rle with old method : 0.1488635540008545 time for calcul the mask position with numpy : 0.06365323066711426 nb_pixel_total : 13981 time to create 1 rle with old method : 0.04114723205566406 time for calcul the mask position with numpy : 0.07420992851257324 nb_pixel_total : 231554 time to create 1 rle with new method : 1.045405387878418 time for calcul the mask position with numpy : 0.06419777870178223 nb_pixel_total : 392446 time to create 1 rle with new method : 0.9891440868377686 create new chi : 9.881853818893433 time to delete rle : 0.002246856689453125 batch 1 Loaded 12 chid ids of type : 3726 Number RLEs to save : 23312 TO DO : save crop sub photo not yet done ! save time : 1.4249796867370605 nb_obj : 13 nb_hashtags : 5 time to prepare the origin masks : 15.596247434616089 time for calcul the mask position with numpy : 0.45479273796081543 nb_pixel_total : 7844453 time to create 1 rle with new method : 0.7102580070495605 time for calcul the mask position with numpy : 0.07101035118103027 nb_pixel_total : 216421 time to create 1 rle with new method : 0.7153434753417969 time for calcul the mask position with numpy : 0.06603693962097168 nb_pixel_total : 395822 time to create 1 rle with new method : 1.0338513851165771 time for calcul the mask position with numpy : 0.057798147201538086 nb_pixel_total : 33254 time to create 1 rle with old method : 0.0369570255279541 time for calcul the mask position with numpy : 0.04364728927612305 nb_pixel_total : 3575 time to create 1 rle with old method : 0.004595041275024414 time for calcul the mask position with numpy : 0.037818193435668945 nb_pixel_total : 11919 time to create 1 rle with old method : 0.039800167083740234 time for calcul the mask position with numpy : 0.07251763343811035 nb_pixel_total : 436580 time to create 1 rle with new method : 1.9725265502929688 time for calcul the mask position with numpy : 0.07099151611328125 nb_pixel_total : 1077736 time to create 1 rle with new method : 0.9892065525054932 time for calcul the mask position with numpy : 0.05544757843017578 nb_pixel_total : 17210 time to create 1 rle with old method : 0.01903700828552246 time for calcul the mask position with numpy : 0.05570626258850098 nb_pixel_total : 33578 time to create 1 rle with old method : 0.03574967384338379 time for calcul the mask position with numpy : 0.05522441864013672 nb_pixel_total : 353265 time to create 1 rle with new method : 0.5879580974578857 time for calcul the mask position with numpy : 0.061005353927612305 nb_pixel_total : 4275 time to create 1 rle with old method : 0.005783796310424805 time for calcul the mask position with numpy : 0.06622552871704102 nb_pixel_total : 98855 time to create 1 rle with old method : 0.10986042022705078 time for calcul the mask position with numpy : 0.07747220993041992 nb_pixel_total : 1803297 time to create 1 rle with new method : 0.6481029987335205 create new chi : 8.479398727416992 time to delete rle : 0.003259420394897461 batch 1 Loaded 14 chid ids of type : 3726 Number RLEs to save : 24253 TO DO : save crop sub photo not yet done ! save time : 1.3922290802001953 nb_obj : 17 nb_hashtags : 3 time to prepare the origin masks : 15.559382915496826 time for calcul the mask position with numpy : 0.3288140296936035 nb_pixel_total : 9559076 time to create 1 rle with new method : 1.2301664352416992 time for calcul the mask position with numpy : 0.06493258476257324 nb_pixel_total : 136302 time to create 1 rle with old method : 0.16875886917114258 time for calcul the mask position with numpy : 0.06326746940612793 nb_pixel_total : 26612 time to create 1 rle with old method : 0.029022693634033203 time for calcul the mask position with numpy : 0.06179642677307129 nb_pixel_total : 151737 time to create 1 rle with new method : 0.87375807762146 time for calcul the mask position with numpy : 0.065338134765625 nb_pixel_total : 60821 time to create 1 rle with old method : 0.10499739646911621 time for calcul the mask position with numpy : 0.06348514556884766 nb_pixel_total : 49353 time to create 1 rle with old method : 0.05479240417480469 time for calcul the mask position with numpy : 0.06194758415222168 nb_pixel_total : 22394 time to create 1 rle with old method : 0.024863243103027344 time for calcul the mask position with numpy : 0.06870698928833008 nb_pixel_total : 839193 time to create 1 rle with new method : 1.1201775074005127 time for calcul the mask position with numpy : 0.036817073822021484 nb_pixel_total : 52334 time to create 1 rle with old method : 0.06696677207946777 time for calcul the mask position with numpy : 0.04561018943786621 nb_pixel_total : 477712 time to create 1 rle with new method : 0.9298701286315918 time for calcul the mask position with numpy : 0.03942084312438965 nb_pixel_total : 14181 time to create 1 rle with old method : 0.017144203186035156 time for calcul the mask position with numpy : 0.03734469413757324 nb_pixel_total : 39830 time to create 1 rle with old method : 0.044031381607055664 time for calcul the mask position with numpy : 0.04532361030578613 nb_pixel_total : 277256 time to create 1 rle with new method : 1.0160455703735352 time for calcul the mask position with numpy : 0.037858009338378906 nb_pixel_total : 404385 time to create 1 rle with new method : 1.7006440162658691 time for calcul the mask position with numpy : 0.037560224533081055 nb_pixel_total : 26974 time to create 1 rle with old method : 0.0287322998046875 time for calcul the mask position with numpy : 0.03593921661376953 nb_pixel_total : 15203 time to create 1 rle with old method : 0.016961336135864258 time for calcul the mask position with numpy : 0.036400556564331055 nb_pixel_total : 68029 time to create 1 rle with old method : 0.0739283561706543 time for calcul the mask position with numpy : 0.04086565971374512 nb_pixel_total : 108848 time to create 1 rle with old method : 0.11805415153503418 create new chi : 9.067750215530396 time to delete rle : 0.0030984878540039062 batch 1 Loaded 18 chid ids of type : 3726 Number RLEs to save : 21572 TO DO : save crop sub photo not yet done ! save time : 1.303206205368042 nb_obj : 6 nb_hashtags : 2 time to prepare the origin masks : 9.38044548034668 time for calcul the mask position with numpy : 1.721613883972168 nb_pixel_total : 8184629 time to create 1 rle with new method : 0.9711184501647949 time for calcul the mask position with numpy : 0.03979325294494629 nb_pixel_total : 138376 time to create 1 rle with old method : 0.17613482475280762 time for calcul the mask position with numpy : 0.037661075592041016 nb_pixel_total : 122093 time to create 1 rle with old method : 0.16604828834533691 time for calcul the mask position with numpy : 0.056946516036987305 nb_pixel_total : 1659079 time to create 1 rle with new method : 1.6010591983795166 time for calcul the mask position with numpy : 0.059403419494628906 nb_pixel_total : 12308 time to create 1 rle with old method : 0.014123201370239258 time for calcul the mask position with numpy : 0.060880184173583984 nb_pixel_total : 5992 time to create 1 rle with old method : 0.0073053836822509766 time for calcul the mask position with numpy : 0.08336710929870605 nb_pixel_total : 2207763 time to create 1 rle with new method : 1.3547232151031494 create new chi : 6.5029613971710205 time to delete rle : 0.0026502609252929688 batch 1 Loaded 7 chid ids of type : 3726 Number RLEs to save : 20403 TO DO : save crop sub photo not yet done ! save time : 1.2358605861663818 map_output_result : {1386038142: (0.0, 'Should be the crop_list due to order', 0.0), 1386038140: (0.0, 'Should be the crop_list due to order', 0.0), 1386038138: (0.0, 'Should be the crop_list due to order', 0.0), 1386038137: (0.0, 'Should be the crop_list due to order', 0.0), 1386038136: (0.0, 'Should be the crop_list due to order', 0.0), 1386037955: (0.0, 'Should be the crop_list due to order', 0.0), 1386037926: (0.0, 'Should be the crop_list due to order', 0.0), 1386037914: (0.0, 'Should be the crop_list due to order', 0.0), 1386037894: (0.0, 'Should be the crop_list due to order', 0.0), 1386037858: (0.0, 'Should be the crop_list due to order', 0.0), 1386037850: (0.0, 'Should be the crop_list due to order', 0.0), 1386035648: (0.0, 'Should be the crop_list due to order', 0.0), 1386035593: (0.0, 'Should be the crop_list due to order', 0.0), 1386035560: (0.0, 'Should be the crop_list due to order', 0.0), 1386035526: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1386038142, 1386038140, 1386038138, 1386038137, 1386038136, 1386037955, 1386037926, 1386037914, 1386037894, 1386037858, 1386037850, 1386035648, 1386035593, 1386035560, 1386035526] Looping around the photos to save general results len do output : 15 /1386038142.Didn't retrieve data . /1386038140.Didn't retrieve data . /1386038138.Didn't retrieve data . /1386038137.Didn't retrieve data . /1386038136.Didn't retrieve data . /1386037955.Didn't retrieve data . /1386037926.Didn't retrieve data . /1386037914.Didn't retrieve data . /1386037894.Didn't retrieve data . /1386037858.Didn't retrieve data . /1386037850.Didn't retrieve data . /1386035648.Didn't retrieve data . /1386035593.Didn't retrieve data . /1386035560.Didn't retrieve data . /1386035526.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038142', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038140', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038138', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038137', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038136', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037955', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037926', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037914', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037894', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037858', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037850', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035648', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035593', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035560', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035526', None, None, None, None, None, '3784202') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 45 time used for this insertion : 0.01588892936706543 save_final save missing photos in datou_result : time spend for datou_step_exec : 304.29074144363403 time spend to save output : 0.04369044303894043 total time spend for step 8 : 304.334431886673 step9:crop_condition Fri Sep 26 10:39:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 15 ! batch 1 Loaded 165 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! About to upload 9 photos upload in portfolio : 4869462 init cache_photo without model_param we have 9 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758875986_4182383 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 9 photos in the portfolio 4869462 time of upload the photos Elapsed time : 5.1520326137542725 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758875997_4182383 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.490807294845581 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758876006_4182383 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.1204710006713867 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 18 About to insert : list_path_to_insert length 18 new photo from crops ! About to upload 18 photos upload in portfolio : 4869462 init cache_photo without model_param we have 18 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758876025_4182383 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 18 photos in the portfolio 4869462 time of upload the photos Elapsed time : 6.788509130477905 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758876038_4182383 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.8964273929595947 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1386038142, 1386038140, 1386038138, 1386038137, 1386038136, 1386037955, 1386037926, 1386037914, 1386037894, 1386037858, 1386037850, 1386035648, 1386035593, 1386035560, 1386035526] Looping around the photos to save general results len do output : 40 /1386322921Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322923Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322924Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322925Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322927Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322928Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322930Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322931Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322932Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322942Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322943Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322944Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322950Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322951Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322952Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322953Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322954Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322987Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322988Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322989Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322990Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322991Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322992Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322993Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322994Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322995Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322996Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322997Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322998Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386322999Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323000Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323001Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323002Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323003Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323004Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323072Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323074Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323075Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323078Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323079Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038142', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038140', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038138', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038137', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038136', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037955', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037926', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037914', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037894', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037858', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037850', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035648', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035593', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035560', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035526', None, None, None, None, None, '3784202') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 135 time used for this insertion : 0.05328822135925293 save_final save missing photos in datou_result : time spend for datou_step_exec : 70.83847284317017 time spend to save output : 0.0567173957824707 total time spend for step 9 : 70.89519023895264 step10:ventilate_hashtags_in_portfolio Fri Sep 26 10:40:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 27268665 get user id for portfolio 27268665 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27268665 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','environnement','flou','kraft','metal','Teint_Dans_La_Masse','plastique','Carton_brun','Carton_gris','papier','cartonnette','autre_refus')) AND mptpi.`min_score`=0.5 To do To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27268665 AND mptpi.`type`=4417 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','environnement','flou','kraft','metal','Teint_Dans_La_Masse','plastique','Carton_brun','Carton_gris','papier','cartonnette','autre_refus')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://marlene.fotonower.com/velours/27268799,27268800,27268801,27268802,27268803,27268804,27268805,27268806,27268807,27268808,27268809,27268810?tags=flou,autre_refus,cartonnette,papier,Teint_Dans_La_Masse,plastique,mal_croppe,Carton_gris,metal,kraft,Carton_brun,environnement Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1386038142, 1386038140, 1386038138, 1386038137, 1386038136, 1386037955, 1386037926, 1386037914, 1386037894, 1386037858, 1386037850, 1386035648, 1386035593, 1386035560, 1386035526] Looping around the photos to save general results len do output : 1 /27268665. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038142', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038140', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038138', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038137', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038136', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037955', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037926', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037914', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037894', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037858', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037850', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035648', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035593', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035560', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035526', None, None, None, None, None, '3784202') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 16 time used for this insertion : 0.019134044647216797 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.4929027557373047 time spend to save output : 0.01940608024597168 total time spend for step 10 : 0.5123088359832764 step11:final Fri Sep 26 10:40:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1386038142: ('0.1350534389767196',), 1386038140: ('0.1350534389767196',), 1386038138: ('0.1350534389767196',), 1386038137: ('0.1350534389767196',), 1386038136: ('0.1350534389767196',), 1386037955: ('0.1350534389767196',), 1386037926: ('0.1350534389767196',), 1386037914: ('0.1350534389767196',), 1386037894: ('0.1350534389767196',), 1386037858: ('0.1350534389767196',), 1386037850: ('0.1350534389767196',), 1386035648: ('0.1350534389767196',), 1386035593: ('0.1350534389767196',), 1386035560: ('0.1350534389767196',), 1386035526: ('0.1350534389767196',)} new output for save of step final : {1386038142: ('0.1350534389767196',), 1386038140: ('0.1350534389767196',), 1386038138: ('0.1350534389767196',), 1386038137: ('0.1350534389767196',), 1386038136: ('0.1350534389767196',), 1386037955: ('0.1350534389767196',), 1386037926: ('0.1350534389767196',), 1386037914: ('0.1350534389767196',), 1386037894: ('0.1350534389767196',), 1386037858: ('0.1350534389767196',), 1386037850: ('0.1350534389767196',), 1386035648: ('0.1350534389767196',), 1386035593: ('0.1350534389767196',), 1386035560: ('0.1350534389767196',), 1386035526: ('0.1350534389767196',)} [1386038142, 1386038140, 1386038138, 1386038137, 1386038136, 1386037955, 1386037926, 1386037914, 1386037894, 1386037858, 1386037850, 1386035648, 1386035593, 1386035560, 1386035526] Looping around the photos to save general results len do output : 15 /1386038142.Didn't retrieve data . /1386038140.Didn't retrieve data . /1386038138.Didn't retrieve data . /1386038137.Didn't retrieve data . /1386038136.Didn't retrieve data . /1386037955.Didn't retrieve data . /1386037926.Didn't retrieve data . /1386037914.Didn't retrieve data . /1386037894.Didn't retrieve data . /1386037858.Didn't retrieve data . /1386037850.Didn't retrieve data . /1386035648.Didn't retrieve data . /1386035593.Didn't retrieve data . /1386035560.Didn't retrieve data . /1386035526.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038142', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038140', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038138', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038137', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038136', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037955', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037926', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037914', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037894', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037858', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037850', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035648', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035593', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035560', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035526', None, None, None, None, None, '3784202') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 45 time used for this insertion : 0.01727437973022461 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.14757347106933594 time spend to save output : 0.01864027976989746 total time spend for step 11 : 0.1662137508392334 step12:velours_tree Fri Sep 26 10:40:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.47103285789489746 time spend to save output : 3.7670135498046875e-05 total time spend for step 12 : 0.4710705280303955 step13:send_mail_cod Fri Sep 26 10:40:43 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin in order to get the selector url, please entre the license of selector results_Auto_P27268665_26-09-2025_10_40_43.pdf 27268787 imagette272687871758876043 27268788 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272687881758876043 27268789 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272687891758876044 27268792 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272687921758876044 27268793 imagette272687931758876045 27268795 imagette272687951758876045 27268796 change filename to text .imagette272687961758876045 27268791 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272687911758876045 27268794 imagette272687941758876047 27268797 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272687971758876047 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=27268665 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://marlene.fotonower.com/velours/27268799,27268800,27268801,27268802,27268803,27268804,27268805,27268806,27268807,27268808,27268809,27268810?tags=flou,autre_refus,cartonnette,papier,Teint_Dans_La_Masse,plastique,mal_croppe,Carton_gris,metal,kraft,Carton_brun,environnement your option no_mail is active, we will not send the real mail to your client args[1386038142] : ((1386038142, -0.32749308007120204, 496442774), (1386038142, -0.09091778290924059, 492688767), '0.1350534389767196') We are sending mail with results at report@fotonower.com args[1386038140] : ((1386038140, -0.07721828556651654, 496442774), (1386038140, -0.7128978207937444, 492688767), '0.1350534389767196') We are sending mail with results at report@fotonower.com args[1386038138] : ((1386038138, 0.015618659104673206, 2107752395), (1386038138, -0.00790061244724607, 492688767), '0.1350534389767196') We are sending mail with results at report@fotonower.com args[1386038137] : ((1386038137, 0.04915520575055445, 2107752395), (1386038137, 0.5687747367354268, 492688767), '0.1350534389767196') We are sending mail with results at report@fotonower.com args[1386038136] : ((1386038136, 0.06489787195107984, 2107752395), (1386038136, -0.21231373955313973, 492688767), '0.1350534389767196') We are sending mail with results at report@fotonower.com args[1386037955] : ((1386037955, -0.18318580689883585, 496442774), (1386037955, -0.4334229204692662, 492688767), '0.1350534389767196') We are sending mail with results at report@fotonower.com args[1386037926] : ((1386037926, -0.26051105595279833, 496442774), (1386037926, -0.028383289366514408, 492688767), '0.1350534389767196') We are sending mail with results at report@fotonower.com args[1386037914] : ((1386037914, -0.3195842336773323, 496442774), (1386037914, -0.1520993329533846, 492688767), '0.1350534389767196') We are sending mail with results at report@fotonower.com args[1386037894] : ((1386037894, 0.06706915059087648, 2107752395), (1386037894, 0.062414320368215766, 492688767), '0.1350534389767196') We are sending mail with results at report@fotonower.com args[1386037858] : ((1386037858, 0.0674030601399124, 2107752395), (1386037858, -0.19305770316564158, 492688767), '0.1350534389767196') We are sending mail with results at report@fotonower.com args[1386037850] : ((1386037850, -0.15819459645595454, 496442774), (1386037850, 0.16683814579050424, 492688767), '0.1350534389767196') We are sending mail with results at report@fotonower.com args[1386035648] : ((1386035648, -0.19377058125567884, 496442774), (1386035648, 0.48755966284071733, 492688767), '0.1350534389767196') We are sending mail with results at report@fotonower.com args[1386035593] : ((1386035593, -0.18819317018630372, 496442774), (1386035593, -0.0118089054542413, 492688767), '0.1350534389767196') We are sending mail with results at report@fotonower.com args[1386035560] : ((1386035560, -0.2218106056430087, 496442774), (1386035560, -0.12018177399552336, 492688767), '0.1350534389767196') We are sending mail with results at report@fotonower.com args[1386035526] : ((1386035526, -0.2988912590715839, 496442774), (1386035526, 0.17590971396293972, 492688767), '0.1350534389767196') We are sending mail with results at report@fotonower.com refus_total : 0.1350534389767196 2022-04-13 10:29:59 0 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27268665_26-09-2025_10_40_43.pdf results_Auto_P27268665_26-09-2025_10_40_43.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27268665_26-09-2025_10_40_43.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4433','27268665','results_Auto_P27268665_26-09-2025_10_40_43.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27268665_26-09-2025_10_40_43.pdf','pdf','','1.37','0.1350534389767196') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1386038142, 1386038140, 1386038138, 1386038137, 1386038136, 1386037955, 1386037926, 1386037914, 1386037894, 1386037858, 1386037850, 1386035648, 1386035593, 1386035560, 1386035526] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038142', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038140', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038138', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038137', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038136', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037955', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037926', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037914', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037894', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037858', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037850', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035648', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035593', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035560', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035526', None, None, None, None, None, '3784202') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 15 time used for this insertion : 0.014682292938232422 save_final save missing photos in datou_result : time spend for datou_step_exec : 12.024173259735107 time spend to save output : 0.01508784294128418 total time spend for step 13 : 12.039261102676392 step14:split_time_score Fri Sep 26 10:40:55 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('14', 61),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 18092025 27268665 Nombre de photos uploadées : 61 / 23040 (0%) 18092025 27268665 Nombre de photos taguées (types de déchets): 0 / 61 (0%) 18092025 27268665 Nombre de photos taguées (volume) : 0 / 61 (0%) elapsed_time : load_data_split_time_score 1.6689300537109375e-06 elapsed_time : order_list_meta_photo_and_scores 6.198883056640625e-06 ????????????????????????????????????????????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0027697086334228516 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.22873950004577637 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.15401314081438286 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26998004_18-09-2025_12_55_19.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26998004 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26998004 AND mptpi.`type`=3726 To do Qualite : 0.15876110650577957 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26999382_18-09-2025_16_28_13.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26999382 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26999382 AND mptpi.`type`=3726 To do Qualite : 0.10113798898534726 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27000904_18-09-2025_16_06_10.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27000904 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27000904 AND mptpi.`type`=3726 To do Qualite : 0.18364316406527295 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27120288_23-09-2025_22_18_47.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27120288 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27120288 AND mptpi.`type`=3726 To do Qualite : 0.05787806829603807 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27124812_23-09-2025_13_23_41.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27124812 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27124812 AND mptpi.`type`=3726 To do Qualite : 0.09959378803985038 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27172550_24-09-2025_13_54_49.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27172550 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27172550 AND mptpi.`type`=3726 To do Qualite : 0.1160095068060618 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27219530_25-09-2025_08_48_58.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27219530 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27219530 AND mptpi.`type`=3726 To do Qualite : 0.10822141120346863 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27268665_26-09-2025_10_40_43.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27268665 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27268665 AND mptpi.`type`=3726 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'18092025': {'nb_upload': 61, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1386038142, 1386038140, 1386038138, 1386038137, 1386038136, 1386037955, 1386037926, 1386037914, 1386037894, 1386037858, 1386037850, 1386035648, 1386035593, 1386035560, 1386035526] Looping around the photos to save general results len do output : 1 /27268665Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038142', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038140', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038138', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038137', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386038136', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037955', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037926', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037914', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037894', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037858', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386037850', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035648', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035593', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035560', None, None, None, None, None, '3784202') ('4433', None, None, None, None, None, None, None, '3784202') ('4433', None, '1386035526', None, None, None, None, None, '3784202') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 16 time used for this insertion : 0.0171053409576416 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.9921891689300537 time spend to save output : 0.017287731170654297 total time spend for step 14 : 1.009476900100708 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 464.23user 312.64system 14:39.78elapsed 88%CPU (0avgtext+0avgdata 5271648maxresident)k 11521176inputs+529384outputs (339257major+39840438minor)pagefaults 0swaps