python /home/admin/mtr/script_for_cron.py -j default -m 20 -a 'python3 ~/workarea/git/Velours/python/prod/datou.py -j batch_current -a 4433' -s traitement_sts -M 0 -S 0 -U 100,80,95 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/home/admin/workarea/git/apy', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 30252 load datou : 4433 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4433, datou_cur_ids : ['3777389'] with mtr_portfolio_ids : ['27224793'] and first list_photo_ids : [] new path : /proc/30252/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, brightness, blur_detection, crop_condition, thcl, tfhub_classification2, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 15 list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 11 ; length of list_pids : 11 ; length of list_args : 11 time to download the photos : 3.432354211807251 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 14 step1:mask_detect Fri Sep 26 11:00:58 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10582 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-09-26 11:01:02.437655: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-26 11:01:02.472516: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-26 11:01:02.474614: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f3c34000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-26 11:01:02.474641: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-26 11:01:02.480307: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-26 11:01:02.681554: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x6dfe5a0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-26 11:01:02.681634: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-26 11:01:02.683122: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:01:02.688671: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:01:02.727204: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:01:02.747299: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:01:02.751261: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:01:02.782942: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:01:02.787386: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:01:02.844831: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:01:02.847156: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:01:02.847640: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:01:02.849723: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-26 11:01:02.849755: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-26 11:01:02.849771: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-26 11:01:02.852526: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9801 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-09-26 11:01:03.188833: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:01:03.188958: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:01:03.188977: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:01:03.188994: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:01:03.189010: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:01:03.189025: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:01:03.189040: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:01:03.189056: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:01:03.190241: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:01:03.191463: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:01:03.191494: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:01:03.191509: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:01:03.191528: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:01:03.191542: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:01:03.191555: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:01:03.191569: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:01:03.191583: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:01:03.192821: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:01:03.192869: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-26 11:01:03.192877: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-26 11:01:03.192885: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-26 11:01:03.194104: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9801 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-09-26 11:01:17.945287: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:01:18.299078: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 11 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 18 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 3.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 16 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 13 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 15 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 15 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 19 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 3.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 15 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 4.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 2.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 15 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 2.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 18 Detection mask done ! Trying to reset tf kernel 30439 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5290 tf kernel not reseted sub process len(results) : 11 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 11 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10582 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.0008542537689208984 nb_pixel_total : 7417 time to create 1 rle with old method : 0.011513233184814453 length of segment : 82 time for calcul the mask position with numpy : 0.0007548332214355469 nb_pixel_total : 25578 time to create 1 rle with old method : 0.029918193817138672 length of segment : 250 time for calcul the mask position with numpy : 0.0004973411560058594 nb_pixel_total : 26676 time to create 1 rle with old method : 0.033635616302490234 length of segment : 247 time for calcul the mask position with numpy : 0.028459787368774414 nb_pixel_total : 708935 time to create 1 rle with new method : 0.050795793533325195 length of segment : 911 time for calcul the mask position with numpy : 0.006745338439941406 nb_pixel_total : 408598 time to create 1 rle with new method : 0.13763856887817383 length of segment : 1021 time for calcul the mask position with numpy : 0.04024839401245117 nb_pixel_total : 955135 time to create 1 rle with new method : 0.07604074478149414 length of segment : 1186 time for calcul the mask position with numpy : 0.0005507469177246094 nb_pixel_total : 16321 time to create 1 rle with old method : 0.019240379333496094 length of segment : 108 time for calcul the mask position with numpy : 0.0020694732666015625 nb_pixel_total : 40600 time to create 1 rle with old method : 0.04615020751953125 length of segment : 496 time for calcul the mask position with numpy : 0.004207134246826172 nb_pixel_total : 150261 time to create 1 rle with new method : 0.006436347961425781 length of segment : 818 time for calcul the mask position with numpy : 0.0009291172027587891 nb_pixel_total : 26144 time to create 1 rle with old method : 0.029504776000976562 length of segment : 264 time for calcul the mask position with numpy : 0.014133453369140625 nb_pixel_total : 615754 time to create 1 rle with new method : 0.04728555679321289 length of segment : 837 time for calcul the mask position with numpy : 0.1891622543334961 nb_pixel_total : 1382145 time to create 1 rle with new method : 0.11883330345153809 length of segment : 2482 time for calcul the mask position with numpy : 0.0004987716674804688 nb_pixel_total : 21950 time to create 1 rle with old method : 0.024289846420288086 length of segment : 241 time for calcul the mask position with numpy : 0.0029532909393310547 nb_pixel_total : 148693 time to create 1 rle with old method : 0.15737438201904297 length of segment : 566 time for calcul the mask position with numpy : 0.0007343292236328125 nb_pixel_total : 17496 time to create 1 rle with old method : 0.01922917366027832 length of segment : 168 time for calcul the mask position with numpy : 0.0016374588012695312 nb_pixel_total : 69090 time to create 1 rle with old method : 0.07701373100280762 length of segment : 288 time for calcul the mask position with numpy : 0.0060787200927734375 nb_pixel_total : 277407 time to create 1 rle with new method : 0.009211063385009766 length of segment : 427 time for calcul the mask position with numpy : 0.004496097564697266 nb_pixel_total : 97352 time to create 1 rle with old method : 0.11134672164916992 length of segment : 441 time for calcul the mask position with numpy : 0.0007703304290771484 nb_pixel_total : 27030 time to create 1 rle with old method : 0.029340505599975586 length of segment : 220 time for calcul the mask position with numpy : 0.006233930587768555 nb_pixel_total : 318252 time to create 1 rle with new method : 0.008548736572265625 length of segment : 700 time for calcul the mask position with numpy : 0.014005899429321289 nb_pixel_total : 577117 time to create 1 rle with new method : 0.04503798484802246 length of segment : 1198 time for calcul the mask position with numpy : 0.04363393783569336 nb_pixel_total : 1098472 time to create 1 rle with new method : 0.20301508903503418 length of segment : 1756 time for calcul the mask position with numpy : 0.0007953643798828125 nb_pixel_total : 29977 time to create 1 rle with old method : 0.0328679084777832 length of segment : 166 time for calcul the mask position with numpy : 0.0060236454010009766 nb_pixel_total : 292918 time to create 1 rle with new method : 0.0088043212890625 length of segment : 529 time for calcul the mask position with numpy : 0.0631096363067627 nb_pixel_total : 1245643 time to create 1 rle with new method : 0.58475661277771 length of segment : 2864 time for calcul the mask position with numpy : 0.022919654846191406 nb_pixel_total : 1186416 time to create 1 rle with new method : 0.10836982727050781 length of segment : 2001 time for calcul the mask position with numpy : 0.010308504104614258 nb_pixel_total : 810729 time to create 1 rle with new method : 0.08614635467529297 length of segment : 1089 time for calcul the mask position with numpy : 0.009980201721191406 nb_pixel_total : 793314 time to create 1 rle with new method : 0.0189058780670166 length of segment : 1066 time for calcul the mask position with numpy : 0.008797168731689453 nb_pixel_total : 566634 time to create 1 rle with new method : 0.021961212158203125 length of segment : 1403 time for calcul the mask position with numpy : 0.003559112548828125 nb_pixel_total : 211778 time to create 1 rle with new method : 0.005258321762084961 length of segment : 620 time for calcul the mask position with numpy : 0.0013325214385986328 nb_pixel_total : 61662 time to create 1 rle with old method : 0.06564569473266602 length of segment : 287 time for calcul the mask position with numpy : 0.0021104812622070312 nb_pixel_total : 96846 time to create 1 rle with old method : 0.10338807106018066 length of segment : 287 time for calcul the mask position with numpy : 0.0013477802276611328 nb_pixel_total : 55452 time to create 1 rle with old method : 0.05903935432434082 length of segment : 252 time for calcul the mask position with numpy : 0.004136562347412109 nb_pixel_total : 221427 time to create 1 rle with new method : 0.008584976196289062 length of segment : 761 time for calcul the mask position with numpy : 0.002469301223754883 nb_pixel_total : 104146 time to create 1 rle with old method : 0.1078195571899414 length of segment : 525 time for calcul the mask position with numpy : 0.009024620056152344 nb_pixel_total : 415390 time to create 1 rle with new method : 0.0166473388671875 length of segment : 1319 time for calcul the mask position with numpy : 0.0002779960632324219 nb_pixel_total : 6142 time to create 1 rle with old method : 0.006645679473876953 length of segment : 118 time for calcul the mask position with numpy : 0.0005445480346679688 nb_pixel_total : 12678 time to create 1 rle with old method : 0.014243364334106445 length of segment : 163 time for calcul the mask position with numpy : 0.00016832351684570312 nb_pixel_total : 4781 time to create 1 rle with old method : 0.005220890045166016 length of segment : 94 time for calcul the mask position with numpy : 0.00028014183044433594 nb_pixel_total : 10650 time to create 1 rle with old method : 0.011960268020629883 length of segment : 94 time for calcul the mask position with numpy : 0.0007114410400390625 nb_pixel_total : 17540 time to create 1 rle with old method : 0.019533395767211914 length of segment : 181 time for calcul the mask position with numpy : 0.05023980140686035 nb_pixel_total : 1936261 time to create 1 rle with new method : 0.41722702980041504 length of segment : 2989 time for calcul the mask position with numpy : 0.0004832744598388672 nb_pixel_total : 14231 time to create 1 rle with old method : 0.015430688858032227 length of segment : 98 time for calcul the mask position with numpy : 0.0005991458892822266 nb_pixel_total : 24670 time to create 1 rle with old method : 0.028019428253173828 length of segment : 140 time for calcul the mask position with numpy : 0.001192331314086914 nb_pixel_total : 55430 time to create 1 rle with old method : 0.0598757266998291 length of segment : 425 time for calcul the mask position with numpy : 0.00040721893310546875 nb_pixel_total : 25495 time to create 1 rle with old method : 0.026999711990356445 length of segment : 271 time for calcul the mask position with numpy : 0.0025377273559570312 nb_pixel_total : 72388 time to create 1 rle with old method : 0.07826685905456543 length of segment : 432 time for calcul the mask position with numpy : 0.0005106925964355469 nb_pixel_total : 17255 time to create 1 rle with old method : 0.01856255531311035 length of segment : 139 time for calcul the mask position with numpy : 0.0048902034759521484 nb_pixel_total : 246522 time to create 1 rle with new method : 0.007236480712890625 length of segment : 630 time for calcul the mask position with numpy : 0.0011682510375976562 nb_pixel_total : 72683 time to create 1 rle with old method : 0.07681059837341309 length of segment : 430 time for calcul the mask position with numpy : 0.015589237213134766 nb_pixel_total : 438432 time to create 1 rle with new method : 0.06284832954406738 length of segment : 1277 time for calcul the mask position with numpy : 0.00263214111328125 nb_pixel_total : 155056 time to create 1 rle with new method : 0.0067980289459228516 length of segment : 602 time for calcul the mask position with numpy : 0.0008003711700439453 nb_pixel_total : 25163 time to create 1 rle with old method : 0.028455734252929688 length of segment : 228 time for calcul the mask position with numpy : 0.002063751220703125 nb_pixel_total : 73715 time to create 1 rle with old method : 0.07923698425292969 length of segment : 351 time for calcul the mask position with numpy : 0.0010132789611816406 nb_pixel_total : 44515 time to create 1 rle with old method : 0.04814004898071289 length of segment : 267 time for calcul the mask position with numpy : 0.00800776481628418 nb_pixel_total : 300408 time to create 1 rle with new method : 0.11817336082458496 length of segment : 550 time for calcul the mask position with numpy : 0.0002460479736328125 nb_pixel_total : 4632 time to create 1 rle with old method : 0.005244016647338867 length of segment : 82 time for calcul the mask position with numpy : 0.06250667572021484 nb_pixel_total : 802024 time to create 1 rle with new method : 0.5938231945037842 length of segment : 2835 time for calcul the mask position with numpy : 0.005294322967529297 nb_pixel_total : 382202 time to create 1 rle with new method : 0.013440370559692383 length of segment : 801 time for calcul the mask position with numpy : 0.0006146430969238281 nb_pixel_total : 33757 time to create 1 rle with old method : 0.03695344924926758 length of segment : 186 time for calcul the mask position with numpy : 0.0004687309265136719 nb_pixel_total : 23748 time to create 1 rle with old method : 0.02552509307861328 length of segment : 266 time for calcul the mask position with numpy : 0.029557228088378906 nb_pixel_total : 1278225 time to create 1 rle with new method : 0.6201503276824951 length of segment : 2237 time for calcul the mask position with numpy : 0.005921363830566406 nb_pixel_total : 443652 time to create 1 rle with new method : 0.014640331268310547 length of segment : 578 time for calcul the mask position with numpy : 0.014161825180053711 nb_pixel_total : 874166 time to create 1 rle with new method : 0.1730804443359375 length of segment : 1546 time for calcul the mask position with numpy : 0.031264543533325195 nb_pixel_total : 1401516 time to create 1 rle with new method : 0.19514942169189453 length of segment : 2414 time for calcul the mask position with numpy : 0.0004057884216308594 nb_pixel_total : 6577 time to create 1 rle with old method : 0.007267951965332031 length of segment : 116 time for calcul the mask position with numpy : 0.0044443607330322266 nb_pixel_total : 192688 time to create 1 rle with new method : 0.006260395050048828 length of segment : 641 time for calcul the mask position with numpy : 0.0035686492919921875 nb_pixel_total : 124804 time to create 1 rle with old method : 0.13586187362670898 length of segment : 512 time for calcul the mask position with numpy : 0.0023651123046875 nb_pixel_total : 78854 time to create 1 rle with old method : 0.08545827865600586 length of segment : 565 time for calcul the mask position with numpy : 0.0004189014434814453 nb_pixel_total : 11140 time to create 1 rle with old method : 0.013448715209960938 length of segment : 146 time for calcul the mask position with numpy : 0.002119302749633789 nb_pixel_total : 64569 time to create 1 rle with old method : 0.07357096672058105 length of segment : 355 time for calcul the mask position with numpy : 0.010297775268554688 nb_pixel_total : 373515 time to create 1 rle with new method : 0.019625425338745117 length of segment : 792 time for calcul the mask position with numpy : 0.0009286403656005859 nb_pixel_total : 28171 time to create 1 rle with old method : 0.03364992141723633 length of segment : 219 time for calcul the mask position with numpy : 0.00919485092163086 nb_pixel_total : 398701 time to create 1 rle with new method : 0.013165473937988281 length of segment : 848 time for calcul the mask position with numpy : 0.008186101913452148 nb_pixel_total : 430228 time to create 1 rle with new method : 0.031523942947387695 length of segment : 806 time for calcul the mask position with numpy : 0.004277706146240234 nb_pixel_total : 191109 time to create 1 rle with new method : 0.006967067718505859 length of segment : 644 time for calcul the mask position with numpy : 0.0011594295501708984 nb_pixel_total : 35933 time to create 1 rle with old method : 0.04521012306213379 length of segment : 215 time for calcul the mask position with numpy : 0.0006320476531982422 nb_pixel_total : 31682 time to create 1 rle with old method : 0.03516364097595215 length of segment : 292 time for calcul the mask position with numpy : 0.002902984619140625 nb_pixel_total : 82882 time to create 1 rle with old method : 0.1159210205078125 length of segment : 811 time for calcul the mask position with numpy : 0.00020122528076171875 nb_pixel_total : 1978 time to create 1 rle with old method : 0.0023212432861328125 length of segment : 58 time for calcul the mask position with numpy : 0.007518768310546875 nb_pixel_total : 411113 time to create 1 rle with new method : 0.017382383346557617 length of segment : 853 time for calcul the mask position with numpy : 0.003286123275756836 nb_pixel_total : 148022 time to create 1 rle with old method : 0.16287016868591309 length of segment : 465 time for calcul the mask position with numpy : 0.003728151321411133 nb_pixel_total : 113087 time to create 1 rle with old method : 0.12674474716186523 length of segment : 354 time for calcul the mask position with numpy : 0.04658651351928711 nb_pixel_total : 1616896 time to create 1 rle with new method : 0.5072615146636963 length of segment : 2501 time for calcul the mask position with numpy : 0.0028820037841796875 nb_pixel_total : 65959 time to create 1 rle with old method : 0.07134890556335449 length of segment : 308 time for calcul the mask position with numpy : 0.0008845329284667969 nb_pixel_total : 32139 time to create 1 rle with old method : 0.035676002502441406 length of segment : 223 time for calcul the mask position with numpy : 0.008119344711303711 nb_pixel_total : 250868 time to create 1 rle with new method : 0.015940189361572266 length of segment : 687 time for calcul the mask position with numpy : 0.00031685829162597656 nb_pixel_total : 6041 time to create 1 rle with old method : 0.00678563117980957 length of segment : 107 time for calcul the mask position with numpy : 0.0010602474212646484 nb_pixel_total : 57121 time to create 1 rle with old method : 0.06547951698303223 length of segment : 173 time for calcul the mask position with numpy : 0.001020669937133789 nb_pixel_total : 70131 time to create 1 rle with old method : 0.07975029945373535 length of segment : 238 time for calcul the mask position with numpy : 0.006468772888183594 nb_pixel_total : 248473 time to create 1 rle with new method : 0.010259866714477539 length of segment : 587 time for calcul the mask position with numpy : 0.005425214767456055 nb_pixel_total : 189985 time to create 1 rle with new method : 0.008508443832397461 length of segment : 680 time for calcul the mask position with numpy : 0.001111745834350586 nb_pixel_total : 39544 time to create 1 rle with old method : 0.04556560516357422 length of segment : 271 time for calcul the mask position with numpy : 0.0007772445678710938 nb_pixel_total : 15801 time to create 1 rle with old method : 0.018923521041870117 length of segment : 257 time for calcul the mask position with numpy : 0.0012750625610351562 nb_pixel_total : 66413 time to create 1 rle with old method : 0.07488155364990234 length of segment : 226 time for calcul the mask position with numpy : 0.000873565673828125 nb_pixel_total : 28067 time to create 1 rle with old method : 0.03228902816772461 length of segment : 199 time for calcul the mask position with numpy : 0.004245281219482422 nb_pixel_total : 267989 time to create 1 rle with new method : 0.009127616882324219 length of segment : 577 time for calcul the mask position with numpy : 0.0038580894470214844 nb_pixel_total : 125315 time to create 1 rle with old method : 0.13878893852233887 length of segment : 409 time for calcul the mask position with numpy : 0.0010325908660888672 nb_pixel_total : 31320 time to create 1 rle with old method : 0.036028385162353516 length of segment : 222 time for calcul the mask position with numpy : 0.000370025634765625 nb_pixel_total : 8568 time to create 1 rle with old method : 0.010256052017211914 length of segment : 86 time for calcul the mask position with numpy : 0.0008509159088134766 nb_pixel_total : 22691 time to create 1 rle with old method : 0.026401996612548828 length of segment : 243 time for calcul the mask position with numpy : 0.012572526931762695 nb_pixel_total : 466075 time to create 1 rle with new method : 0.025933504104614258 length of segment : 762 time for calcul the mask position with numpy : 0.01590728759765625 nb_pixel_total : 686200 time to create 1 rle with new method : 0.07307767868041992 length of segment : 1173 time for calcul the mask position with numpy : 0.009730815887451172 nb_pixel_total : 660560 time to create 1 rle with new method : 0.14238214492797852 length of segment : 1070 time for calcul the mask position with numpy : 0.0006361007690429688 nb_pixel_total : 25259 time to create 1 rle with old method : 0.029175281524658203 length of segment : 131 time for calcul the mask position with numpy : 0.001107931137084961 nb_pixel_total : 36817 time to create 1 rle with old method : 0.04007768630981445 length of segment : 243 time for calcul the mask position with numpy : 0.0005731582641601562 nb_pixel_total : 15658 time to create 1 rle with old method : 0.017497539520263672 length of segment : 169 time for calcul the mask position with numpy : 0.0025746822357177734 nb_pixel_total : 101628 time to create 1 rle with old method : 0.11458802223205566 length of segment : 337 time for calcul the mask position with numpy : 0.027833223342895508 nb_pixel_total : 1066363 time to create 1 rle with new method : 0.11666679382324219 length of segment : 2807 time for calcul the mask position with numpy : 0.002309560775756836 nb_pixel_total : 115850 time to create 1 rle with old method : 0.12320923805236816 length of segment : 455 time for calcul the mask position with numpy : 0.002803325653076172 nb_pixel_total : 23496 time to create 1 rle with old method : 0.026154518127441406 length of segment : 237 time for calcul the mask position with numpy : 0.002622365951538086 nb_pixel_total : 90884 time to create 1 rle with old method : 0.09905338287353516 length of segment : 471 time for calcul the mask position with numpy : 0.005150794982910156 nb_pixel_total : 185264 time to create 1 rle with new method : 0.009493112564086914 length of segment : 485 time for calcul the mask position with numpy : 0.0021500587463378906 nb_pixel_total : 99590 time to create 1 rle with old method : 0.10741209983825684 length of segment : 443 time for calcul the mask position with numpy : 0.0023632049560546875 nb_pixel_total : 102624 time to create 1 rle with old method : 0.111724853515625 length of segment : 386 time for calcul the mask position with numpy : 0.0034134387969970703 nb_pixel_total : 136708 time to create 1 rle with old method : 0.14975523948669434 length of segment : 399 time for calcul the mask position with numpy : 0.0009486675262451172 nb_pixel_total : 44073 time to create 1 rle with old method : 0.04665780067443848 length of segment : 183 time for calcul the mask position with numpy : 0.008396625518798828 nb_pixel_total : 389774 time to create 1 rle with new method : 0.021291255950927734 length of segment : 806 time for calcul the mask position with numpy : 0.0064580440521240234 nb_pixel_total : 274366 time to create 1 rle with new method : 0.009361743927001953 length of segment : 660 time for calcul the mask position with numpy : 0.0006842613220214844 nb_pixel_total : 28170 time to create 1 rle with old method : 0.031568050384521484 length of segment : 210 time for calcul the mask position with numpy : 0.0024220943450927734 nb_pixel_total : 109371 time to create 1 rle with old method : 0.12300729751586914 length of segment : 270 time for calcul the mask position with numpy : 0.0007481575012207031 nb_pixel_total : 23918 time to create 1 rle with old method : 0.027601957321166992 length of segment : 141 time for calcul the mask position with numpy : 0.004598379135131836 nb_pixel_total : 161454 time to create 1 rle with new method : 0.006730318069458008 length of segment : 421 time for calcul the mask position with numpy : 0.0008039474487304688 nb_pixel_total : 15363 time to create 1 rle with old method : 0.01732659339904785 length of segment : 229 time for calcul the mask position with numpy : 0.0006537437438964844 nb_pixel_total : 31391 time to create 1 rle with old method : 0.03597617149353027 length of segment : 119 time for calcul the mask position with numpy : 0.0016598701477050781 nb_pixel_total : 35570 time to create 1 rle with old method : 0.041086435317993164 length of segment : 228 time for calcul the mask position with numpy : 0.0028977394104003906 nb_pixel_total : 95032 time to create 1 rle with old method : 0.10609602928161621 length of segment : 523 time for calcul the mask position with numpy : 0.04340958595275879 nb_pixel_total : 1145927 time to create 1 rle with new method : 0.24601173400878906 length of segment : 2456 time for calcul the mask position with numpy : 0.0005047321319580078 nb_pixel_total : 12294 time to create 1 rle with old method : 0.013520479202270508 length of segment : 178 time for calcul the mask position with numpy : 0.011806249618530273 nb_pixel_total : 454331 time to create 1 rle with new method : 0.017037391662597656 length of segment : 828 time for calcul the mask position with numpy : 0.0032281875610351562 nb_pixel_total : 67770 time to create 1 rle with old method : 0.07459712028503418 length of segment : 370 time for calcul the mask position with numpy : 0.0018849372863769531 nb_pixel_total : 47072 time to create 1 rle with old method : 0.05442619323730469 length of segment : 489 time for calcul the mask position with numpy : 0.0019712448120117188 nb_pixel_total : 69503 time to create 1 rle with old method : 0.07786846160888672 length of segment : 336 time spent for convertir_results : 27.990795850753784 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 141 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 83001 save missing photos in datou_result : time spend for datou_step_exec : 94.08588528633118 time spend to save output : 4.923081874847412 total time spend for step 1 : 99.00896716117859 step2:brightness Fri Sep 26 11:02:37 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness toutes les photos sont déjà traitées, on saute les calculs Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 11 time used for this insertion : 0.009748697280883789 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 11 time used for this insertion : 0.008899450302124023 save missing photos in datou_result : time spend for datou_step_exec : 0.03150010108947754 time spend to save output : 0.02289748191833496 total time spend for step 2 : 0.0543975830078125 step3:blur_detection Fri Sep 26 11:02:37 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection toutes les photos sont déjà traitées, on saute les calculs Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 11 time used for this insertion : 0.00974130630493164 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 11 time used for this insertion : 0.009027481079101562 save missing photos in datou_result : time spend for datou_step_exec : 0.021366596221923828 time spend to save output : 0.023520946502685547 total time spend for step 3 : 0.044887542724609375 step4:crop_condition Fri Sep 26 11:02:37 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 11 ! batch 1 Loaded 141 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 13 About to insert : list_path_to_insert length 13 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 26 About to insert : list_path_to_insert length 26 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 31 About to insert : list_path_to_insert length 31 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 13 About to insert : list_path_to_insert length 13 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1386056377, 1386056344, 1386056336, 1386056328, 1386056324, 1386055709, 1386055705, 1386055687, 1386055658, 1386055654, 1386055621] Looping around the photos to save general results len do output : 95 /-3974581457Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581482Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581489Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581500Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581507Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581551Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581553Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581577Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581465Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581473Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581480Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581502Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581504Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581501Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581499Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581514Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581508Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581527Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581550Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581571Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581564Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581486Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581541Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581567Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581563Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581460Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581459Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581481Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581478Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581472Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581487Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581505Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581515Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581512Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581506Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581520Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581519Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581525Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581531Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581533Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581535Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976615276Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581547Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581548Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581552Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581558Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581566Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581569Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581578Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581580Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581573Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581454Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581455Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581458Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581452Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581453Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581467Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581471Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581469Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581470Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581475Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581468Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581485Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581483Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581497Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581495Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581498Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976615275Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581509Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581511Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581521Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581518Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581545Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581565Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581559Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581561Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581560Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581576Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581572Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581575Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581574Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976615278Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581456Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581476Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581479Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581484Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581488Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581503Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581516Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581526Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581528Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581543Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581554Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581555Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3974581583Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056377', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056344', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056336', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056328', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056324', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055709', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055705', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055687', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055658', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055654', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055621', None, None, None, None, None, '3777389') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 296 time used for this insertion : 0.038747549057006836 save_final save missing photos in datou_result : time spend for datou_step_exec : 109.03734564781189 time spend to save output : 0.051549434661865234 total time spend for step 4 : 109.08889508247375 step5:thcl Fri Sep 26 11:04:26 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 1 we are using the classfication for multi_thcl [2456] time to import caffe and check if the image exist : 0.016388416290283203 time to convert the images to numpy array : 0.6593611240386963 time to import caffe and check if the image exist : 0.020600318908691406 time to convert the images to numpy array : 0.6860828399658203 time to import caffe and check if the image exist : 0.018660306930541992 time to convert the images to numpy array : 0.7251629829406738 time to import caffe and check if the image exist : 0.0803060531616211 time to convert the images to numpy array : 0.6687440872192383 time to import caffe and check if the image exist : 0.014114856719970703 time to convert the images to numpy array : 0.8626987934112549 time to import caffe and check if the image exist : 0.08350753784179688 time to convert the images to numpy array : 0.9266057014465332 time to import caffe and check if the image exist : 0.021017074584960938 time to convert the images to numpy array : 1.2452805042266846 time to import caffe and check if the image exist : 0.01680779457092285 time to convert the images to numpy array : 1.432990550994873 time to import caffe and check if the image exist : 0.018552064895629883 time to convert the images to numpy array : 1.7319839000701904 time to import caffe and check if the image exist : 0.020853042602539062 time to convert the images to numpy array : 1.8361821174621582 total time to convert the images to numpy array : 2.4523322582244873 list photo_ids error: [] list photo_ids correct : [-3974581535, -3976615276, -3974581547, -3974581548, -3974581552, -3974581558, -3974581566, -3974581569, -3974581578, -3974581580, -3974581475, -3974581468, -3974581485, -3974581483, -3974581497, -3974581495, -3974581498, -3976615275, -3974581509, -3974581511, -3974581573, -3974581454, -3974581455, -3974581458, -3974581452, -3974581453, -3974581467, -3974581471, -3974581469, -3974581470, -3974581521, -3974581518, -3974581545, -3974581565, -3974581559, -3974581561, -3974581560, -3974581576, -3974581572, -3974581575, -3974581528, -3974581543, -3974581554, -3974581555, -3974581583, -3974581487, -3974581505, -3974581515, -3974581512, -3974581506, -3974581520, -3974581519, -3974581525, -3974581531, -3974581533, -3974581457, -3974581482, -3974581489, -3974581500, -3974581507, -3974581551, -3974581553, -3974581577, -3974581465, -3974581473, -3974581564, -3974581486, -3974581541, -3974581567, -3974581563, -3974581460, -3974581459, -3974581481, -3974581478, -3974581472, -3974581574, -3976615278, -3974581456, -3974581476, -3974581479, -3974581484, -3974581488, -3974581503, -3974581516, -3974581526, -3974581480, -3974581502, -3974581504, -3974581501, -3974581499, -3974581514, -3974581508, -3974581527, -3974581550, -3974581571] number of photos to traite : 95 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10582 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/:/home/admin/workarea/git/apy/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10360 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 2.787001848220825 time used to do the prediction : 0.26680779457092285 we don't save the descriptors for this thcl 2456 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 2.0503997802734375e-05 save missing photos in datou_result : time spend for datou_step_exec : 9.447351932525635 time spend to save output : 0.0003275871276855469 total time spend for step 5 : 9.44767951965332 step6:tfhub_classification2 Fri Sep 26 11:04:36 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou_step TFHub with tf2 ! we are using the classfication for only one thcl 3501 begin to check gpu status inside check gpu memory 2025-09-26 11:04:40.055920: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-26 11:04:40.057536: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:04:40.057611: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:04:40.057665: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:04:40.076627: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:04:40.076732: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:04:40.109696: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:04:40.114531: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:04:40.171236: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:04:40.173159: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:04:40.174130: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-26 11:04:40.208602: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-26 11:04:40.210938: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f3b8c000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-26 11:04:40.210974: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-26 11:04:40.217517: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x6162ab0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-26 11:04:40.217563: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-26 11:04:40.219117: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:04:40.219212: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:04:40.219237: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:04:40.219341: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:04:40.219377: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:04:40.219428: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:04:40.219466: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:04:40.219501: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:04:40.221277: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:04:40.221726: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:04:40.221784: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-26 11:04:40.221797: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-26 11:04:40.221807: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-26 11:04:40.223589: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3096 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) l 3637 free memory gpu now : 10358 max_wait_temp : 1 max_wait : 5 1 Physical GPUs, 1 Logical GPUs tagging for thcl : 3501 To do loadFromThcl(), then load ParamDescType : thcl3501 thcls : [{'id': 3501, 'mtr_user_id': 31, 'name': 'learn_papier_mixte_centre_tf_b4_220223_3', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '9087724,9087727,9087725,9087729,9087726,9087723,9087722,9087720,9087721,9087728', 'photo_hashtag_type': 4539, 'photo_desc_type': 5759, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 3501, 'mtr_user_id': 31, 'name': 'learn_papier_mixte_centre_tf_b4_220223_3', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '9087724,9087727,9087725,9087729,9087726,9087723,9087722,9087720,9087721,9087728', 'photo_hashtag_type': 4539, 'photo_desc_type': 5759, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5759 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5759, 'learn_papier_mixte_centre_tf_b4_220223_3', 1792, 1792, 'learn_papier_mixte_centre_tf_b4_220223_3', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 2, 22, 13, 54, 32), datetime.datetime(2023, 2, 22, 13, 54, 32)) model_name : learn_papier_mixte_centre_tf_b4_220223_3 model_param file didn't exist model_name : learn_papier_mixte_centre_tf_b4_220223_3 model_type : tf_classification2 list file need : ['Confusion_Matrix.png', 'Precision_Recall_Carton_brun.jpg', 'Precision_Recall_Carton_gris.jpg', 'Precision_Recall_Teint_Dans_La_Masse.jpg', 'Precision_Recall_autre_refus.jpg', 'Precision_Recall_cartonnette.jpg', 'Precision_Recall_environnement.jpg', 'Precision_Recall_kraft.jpg', 'Precision_Recall_metal.jpg', 'Precision_Recall_papier.jpg', 'Precision_Recall_plastique.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file exist in s3 : ['Confusion_Matrix.png', 'Precision_Recall_Carton_brun.jpg', 'Precision_Recall_Carton_gris.jpg', 'Precision_Recall_Teint_Dans_La_Masse.jpg', 'Precision_Recall_autre_refus.jpg', 'Precision_Recall_cartonnette.jpg', 'Precision_Recall_environnement.jpg', 'Precision_Recall_kraft.jpg', 'Precision_Recall_metal.jpg', 'Precision_Recall_papier.jpg', 'Precision_Recall_plastique.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file manque in s3 : [] local folder : /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3 /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Confusion_Matrix.png size_local : 121269 size in s3 : 121269 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:17 Confusion_Matrix.png already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_Carton_brun.jpg size_local : 76104 size in s3 : 76104 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_Carton_brun.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_Carton_gris.jpg size_local : 82912 size in s3 : 82912 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_Carton_gris.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_Teint_Dans_La_Masse.jpg size_local : 75423 size in s3 : 75423 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:25 Precision_Recall_Teint_Dans_La_Masse.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_autre_refus.jpg size_local : 78883 size in s3 : 78883 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_autre_refus.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_cartonnette.jpg size_local : 81791 size in s3 : 81791 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:30 Precision_Recall_cartonnette.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_environnement.jpg size_local : 59936 size in s3 : 59936 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:17 Precision_Recall_environnement.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_kraft.jpg size_local : 80280 size in s3 : 80280 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:25 Precision_Recall_kraft.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_metal.jpg size_local : 77736 size in s3 : 77736 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_metal.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_papier.jpg size_local : 69041 size in s3 : 69041 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_papier.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_plastique.jpg size_local : 70770 size in s3 : 70770 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:17 Precision_Recall_plastique.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Result_Summary.txt size_local : 1092 size in s3 : 1092 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Result_Summary.txt already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/checkpoint size_local : 99 size in s3 : 99 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 checkpoint already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_checkpoint.ckpt.data-00000-of-00002 size_local : 429801 size in s3 : 429801 create time local : 2023-03-03 12:26:16 create time in s3 : 2023-02-22 12:54:30 model_checkpoint.ckpt.data-00000-of-00002 already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_checkpoint.ckpt.data-00001-of-00002 size_local : 141033188 size in s3 : 141033188 create time local : 2023-03-03 12:26:17 create time in s3 : 2023-02-22 12:54:25 model_checkpoint.ckpt.data-00001-of-00002 already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_checkpoint.ckpt.index size_local : 85618 size in s3 : 85618 create time local : 2023-03-03 12:26:17 create time in s3 : 2023-02-22 12:54:30 model_checkpoint.ckpt.index already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_weights.h5 size_local : 71289448 size in s3 : 71289448 create time local : 2023-03-03 12:26:18 create time in s3 : 2023-02-22 12:54:17 model_weights.h5 already exist and didn't need to update desc size : 1792 Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= module (KerasLayer) (None, 1792) 17673816 _________________________________________________________________ learn_papier_mixte_centre_tf (None, 10) 17930 ================================================================= Total params: 17,691,746 Trainable params: 17,930 Non-trainable params: 17,673,816 _________________________________________________________________ Loading Weights... time used to create the model : 17.72002148628235 time used to load_weights : 0.6371865272521973 0it [00:00, ?it/s] 19it [00:00, 188.73it/s] 49it [00:00, 250.00it/s] 88it [00:00, 307.39it/s]temp/1758877255_30252_1386056377_ec8b47756f7d27949e9ff0e333886aba_rle_crop_3974581457_0.png temp/1758877255_30252_1386056336_3816e0dbdc2c6aa33bea33d32cfc7998_rle_crop_3974581482_0.png temp/1758877255_30252_1386056336_3816e0dbdc2c6aa33bea33d32cfc7998_rle_crop_3974581489_0.png temp/1758877255_30252_1386056328_da9385e9fabc692cf1bba28ee91ff88d_rle_crop_3974581500_0.png temp/1758877255_30252_1386056324_b150fca9e4d80e0c95584b77a31433f5_rle_crop_3974581507_0.png temp/1758877255_30252_1386055658_771aad56b910927029450ee1e25fc464_rle_crop_3974581551_0.png temp/1758877255_30252_1386055658_771aad56b910927029450ee1e25fc464_rle_crop_3974581553_0.png temp/1758877255_30252_1386055621_283c197630380bb05b6df7b7e91f9a66_rle_crop_3974581577_0.png temp/1758877255_30252_1386056377_ec8b47756f7d27949e9ff0e333886aba_rle_crop_3974581465_0.png temp/1758877255_30252_1386056344_d80e7dbbd47b8e846b268a2399f1fecc_rle_crop_3974581473_0.png temp/1758877255_30252_1386056344_d80e7dbbd47b8e846b268a2399f1fecc_rle_crop_3974581480_0.png temp/1758877255_30252_1386056328_da9385e9fabc692cf1bba28ee91ff88d_rle_crop_3974581502_0.png temp/1758877255_30252_1386056328_da9385e9fabc692cf1bba28ee91ff88d_rle_crop_3974581504_0.png temp/1758877255_30252_1386056328_da9385e9fabc692cf1bba28ee91ff88d_rle_crop_3974581501_0.png temp/1758877255_30252_1386056328_da9385e9fabc692cf1bba28ee91ff88d_rle_crop_3974581499_0.png temp/1758877255_30252_1386056324_b150fca9e4d80e0c95584b77a31433f5_rle_crop_3974581514_0.png temp/1758877255_30252_1386056324_b150fca9e4d80e0c95584b77a31433f5_rle_crop_3974581508_0.png temp/1758877255_30252_1386055709_fcb640c60dad2a98fff980d2e347b63e_rle_crop_3974581527_0.png temp/1758877255_30252_1386055687_ecae5bd75ee8753561eba5f7856ef8f4_rle_crop_3974581550_0.png temp/1758877255_30252_1386055654_d616b14297b627c878c936b4ff814db7_rle_crop_3974581571_0.png temp/1758877255_30252_1386055654_d616b14297b627c878c936b4ff814db7_rle_crop_3974581564_0.png temp/1758877255_30252_1386056336_3816e0dbdc2c6aa33bea33d32cfc7998_rle_crop_3974581486_0.png temp/1758877255_30252_1386055705_8fb48625c1e51ff3c7ff298e5c27856d_rle_crop_3974581541_0.png temp/1758877255_30252_1386055654_d616b14297b627c878c936b4ff814db7_rle_crop_3974581567_0.png temp/1758877255_30252_1386055654_d616b14297b627c878c936b4ff814db7_rle_crop_3974581563_0.png temp/1758877255_30252_1386056377_ec8b47756f7d27949e9ff0e333886aba_rle_crop_3974581460_0.png temp/1758877255_30252_1386056377_ec8b47756f7d27949e9ff0e333886aba_rle_crop_3974581459_0.png temp/1758877255_30252_1386056344_d80e7dbbd47b8e846b268a2399f1fecc_rle_crop_3974581481_0.png temp/1758877255_30252_1386056344_d80e7dbbd47b8e846b268a2399f1fecc_rle_crop_3974581478_0.png temp/1758877255_30252_1386056344_d80e7dbbd47b8e846b268a2399f1fecc_rle_crop_3974581472_0.png temp/1758877255_30252_1386056336_3816e0dbdc2c6aa33bea33d32cfc7998_rle_crop_3974581487_0.png temp/1758877255_30252_1386056328_da9385e9fabc692cf1bba28ee91ff88d_rle_crop_3974581505_0.png temp/1758877255_30252_1386056324_b150fca9e4d80e0c95584b77a31433f5_rle_crop_3974581515_0.png temp/1758877255_30252_1386056324_b150fca9e4d80e0c95584b77a31433f5_rle_crop_3974581512_0.png temp/1758877255_30252_1386056324_b150fca9e4d80e0c95584b77a31433f5_rle_crop_3974581506_0.png temp/1758877255_30252_1386055709_fcb640c60dad2a98fff980d2e347b63e_rle_crop_3974581520_0.png temp/1758877255_30252_1386055709_fcb640c60dad2a98fff980d2e347b63e_rle_crop_3974581519_0.png temp/1758877255_30252_1386055709_fcb640c60dad2a98fff980d2e347b63e_rle_crop_3974581525_0.png temp/1758877255_30252_1386055709_fcb640c60dad2a98fff980d2e347b63e_rle_crop_3974581531_0.png temp/1758877255_30252_1386055709_fcb640c60dad2a98fff980d2e347b63e_rle_crop_3974581533_0.png temp/1758877255_30252_1386055705_8fb48625c1e51ff3c7ff298e5c27856d_rle_crop_3974581535_0.png temp/1758877255_30252_1386055705_8fb48625c1e51ff3c7ff298e5c27856d_rle_crop_3976615276_0.png temp/1758877255_30252_1386055687_ecae5bd75ee8753561eba5f7856ef8f4_rle_crop_3974581547_0.png temp/1758877255_30252_1386055687_ecae5bd75ee8753561eba5f7856ef8f4_rle_crop_3974581548_0.png temp/1758877255_30252_1386055658_771aad56b910927029450ee1e25fc464_rle_crop_3974581552_0.png temp/1758877255_30252_1386055658_771aad56b910927029450ee1e25fc464_rle_crop_3974581558_0.png temp/1758877255_30252_1386055654_d616b14297b627c878c936b4ff814db7_rle_crop_3974581566_0.png temp/1758877255_30252_1386055654_d616b14297b627c878c936b4ff814db7_rle_crop_3974581569_0.png temp/1758877255_30252_1386055621_283c197630380bb05b6df7b7e91f9a66_rle_crop_3974581578_0.png temp/1758877255_30252_1386055621_283c197630380bb05b6df7b7e91f9a66_rle_crop_3974581580_0.png temp/1758877255_30252_1386055621_283c197630380bb05b6df7b7e91f9a66_rle_crop_3974581573_0.png temp/1758877255_30252_1386056377_ec8b47756f7d27949e9ff0e333886aba_rle_crop_3974581454_0.png temp/1758877255_30252_1386056377_ec8b47756f7d27949e9ff0e333886aba_rle_crop_3974581455_0.png temp/1758877255_30252_1386056377_ec8b47756f7d27949e9ff0e333886aba_rle_crop_3974581458_0.png temp/1758877255_30252_1386056377_ec8b47756f7d27949e9ff0e333886aba_rle_crop_3974581452_0.png temp/1758877255_30252_1386056377_ec8b47756f7d27949e9ff0e333886aba_rle_crop_3974581453_0.png temp/1758877255_30252_1386056344_d80e7dbbd47b8e846b268a2399f1fecc_rle_crop_3974581467_0.png temp/1758877255_30252_1386056344_d80e7dbbd47b8e846b268a2399f1fecc_rle_crop_3974581471_0.png temp/1758877255_30252_1386056344_d80e7dbbd47b8e846b268a2399f1fecc_rle_crop_3974581469_0.png temp/1758877255_30252_1386056344_d80e7dbbd47b8e846b268a2399f1fecc_rle_crop_3974581470_0.png temp/1758877255_30252_1386056344_d80e7dbbd47b8e846b268a2399f1fecc_rle_crop_3974581475_0.png temp/1758877255_30252_1386056344_d80e7dbbd47b8e846b268a2399f1fecc_rle_crop_3974581468_0.png temp/1758877255_30252_1386056336_3816e0dbdc2c6aa33bea33d32cfc7998_rle_crop_3974581485_0.png temp/1758877255_30252_1386056336_3816e0dbdc2c6aa33bea33d32cfc7998_rle_crop_3974581483_0.png temp/1758877255_30252_1386056328_da9385e9fabc692cf1bba28ee91ff88d_rle_crop_3974581497_0.png temp/1758877255_30252_1386056328_da9385e9fabc692cf1bba28ee91ff88d_rle_crop_3974581495_0.png temp/1758877255_30252_1386056328_da9385e9fabc692cf1bba28ee91ff88d_rle_crop_3974581498_0.png temp/1758877255_30252_1386056324_b150fca9e4d80e0c95584b77a31433f5_rle_crop_3976615275_0.png temp/1758877255_30252_1386056324_b150fca9e4d80e0c95584b77a31433f5_rle_crop_3974581509_0.png temp/1758877255_30252_1386056324_b150fca9e4d80e0c95584b77a31433f5_rle_crop_3974581511_0.png temp/1758877255_30252_1386055709_fcb640c60dad2a98fff980d2e347b63e_rle_crop_3974581521_0.png temp/1758877255_30252_1386055709_fcb640c60dad2a98fff980d2e347b63e_rle_crop_3974581518_0.png temp/1758877255_30252_1386055687_ecae5bd75ee8753561eba5f7856ef8f4_rle_crop_3974581545_0.png temp/1758877255_30252_1386055654_d616b14297b627c878c936b4ff814db7_rle_crop_3974581565_0.png temp/1758877255_30252_1386055654_d616b14297b627c878c936b4ff814db7_rle_crop_3974581559_0.png temp/1758877255_30252_1386055654_d616b14297b627c878c936b4ff814db7_rle_crop_3974581561_0.png temp/1758877255_30252_1386055654_d616b14297b627c878c936b4ff814db7_rle_crop_3974581560_0.png temp/1758877255_30252_1386055621_283c197630380bb05b6df7b7e91f9a66_rle_crop_3974581576_0.png temp/1758877255_30252_1386055621_283c197630380bb05b6df7b7e91f9a66_rle_crop_3974581572_0.png temp/1758877255_30252_1386055621_283c197630380bb05b6df7b7e91f9a66_rle_crop_3974581575_0.png temp/1758877255_30252_1386055621_283c197630380bb05b6df7b7e91f9a66_rle_crop_3974581574_0.png temp/1758877255_30252_1386055621_283c197630380bb05b6df7b7e91f9a66_rle_crop_3976615278_0.png temp/1758877255_30252_1386056377_ec8b47756f7d27949e9ff0e333886aba_rle_crop_3974581456_0.png temp/1758877255_30252_1386056344_d80e7dbbd47b8e846b268a2399f1fecc_rle_crop_3974581476_0.png temp/1758877255_30252_1386056344_d80e7dbbd47b8e846b268a2399f1fecc_rle_crop_3974581479_0.png temp/1758877255_30252_1386056336_3816e0dbdc2c6aa33bea33d32cfc7998_rle_crop_3974581484_0.png temp/1758877255_30252_1386056336_3816e0dbdc2c6aa33bea33d32cfc7998_rle_crop_3974581488_0.png temp/1758877255_30252_1386056328_da9385e9fabc692cf1bba28ee91ff88d_rle_crop_3974581503_0.png temp/1758877255_30252_1386056324_b150fca9e4d80e0c95584b77a31433f5_rle_crop_3974581516_0.png 95it [00:00, 276.54it/s]2025-09-26 11:05:02.164169: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:05:02.198839: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 temp/1758877255_30252_1386055709_fcb640c60dad2a98fff980d2e347b63e_rle_crop_3974581526_0.png temp/1758877255_30252_1386055709_fcb640c60dad2a98fff980d2e347b63e_rle_crop_3974581528_0.png temp/1758877255_30252_1386055705_8fb48625c1e51ff3c7ff298e5c27856d_rle_crop_3974581543_0.png temp/1758877255_30252_1386055658_771aad56b910927029450ee1e25fc464_rle_crop_3974581554_0.png temp/1758877255_30252_1386055658_771aad56b910927029450ee1e25fc464_rle_crop_3974581555_0.png temp/1758877255_30252_1386055621_283c197630380bb05b6df7b7e91f9a66_rle_crop_3974581583_0.png Found 95 images belonging to 1 classes. begin to do the prediction : time used to do the prediction : 7.057322978973389 (95,) (95, 10) (95, 1792) shape of features : (95, 1792) shape of new features : (1, 95, 1792) save descriptor for thcl : 3501 time to traite the descriptors : 0.514218807220459 storage_type for insertDescriptorsMulti : 3 Missing photo l117 : -3974581457 Missing photo l117 : -3974581482 Missing photo l117 : -3974581489 Missing photo l117 : -3974581500 Missing photo l117 : -3974581507 Missing photo l117 : -3974581551 Missing photo l117 : -3974581553 Missing photo l117 : -3974581577 Missing photo l117 : -3974581465 Missing photo l117 : -3974581473 Missing photo l117 : -3974581480 Missing photo l117 : -3974581502 Missing photo l117 : -3974581504 Missing photo l117 : -3974581501 Missing photo l117 : -3974581499 Missing photo l117 : -3974581514 Missing photo l117 : -3974581508 Missing photo l117 : -3974581527 Missing photo l117 : -3974581550 Missing photo l117 : -3974581571 Missing photo l117 : -3974581564 Missing photo l117 : -3974581486 Missing photo l117 : -3974581541 Missing photo l117 : -3974581567 Missing photo l117 : -3974581563 Missing photo l117 : -3974581460 Missing photo l117 : -3974581459 Missing photo l117 : -3974581481 Missing photo l117 : -3974581478 Missing photo l117 : -3974581472 Missing photo l117 : -3974581487 Missing photo l117 : -3974581505 Missing photo l117 : -3974581515 Missing photo l117 : -3974581512 Missing photo l117 : -3974581506 Missing photo l117 : -3974581520 Missing photo l117 : -3974581519 Missing photo l117 : -3974581525 Missing photo l117 : -3974581531 Missing photo l117 : -3974581533 Missing photo l117 : -3974581535 Missing photo l117 : -3976615276 Missing photo l117 : -3974581547 Missing photo l117 : -3974581548 Missing photo l117 : -3974581552 Missing photo l117 : -3974581558 Missing photo l117 : -3974581566 Missing photo l117 : -3974581569 Missing photo l117 : -3974581578 Missing photo l117 : -3974581580 Missing photo l117 : -3974581573 Missing photo l117 : -3974581454 Missing photo l117 : -3974581455 Missing photo l117 : -3974581458 Missing photo l117 : -3974581452 Missing photo l117 : -3974581453 Missing photo l117 : -3974581467 Missing photo l117 : -3974581471 Missing photo l117 : -3974581469 Missing photo l117 : -3974581470 Missing photo l117 : -3974581475 Missing photo l117 : -3974581468 Missing photo l117 : -3974581485 Missing photo l117 : -3974581483 Missing photo l117 : -3974581497 Missing photo l117 : -3974581495 Missing photo l117 : -3974581498 Missing photo l117 : -3976615275 Missing photo l117 : -3974581509 Missing photo l117 : -3974581511 Missing photo l117 : -3974581521 Missing photo l117 : -3974581518 Missing photo l117 : -3974581545 Missing photo l117 : -3974581565 Missing photo l117 : -3974581559 Missing photo l117 : -3974581561 Missing photo l117 : -3974581560 Missing photo l117 : -3974581576 Missing photo l117 : -3974581572 Missing photo l117 : -3974581575 Missing photo l117 : -3974581574 Missing photo l117 : -3976615278 Missing photo l117 : -3974581456 Missing photo l117 : -3974581476 Missing photo l117 : -3974581479 Missing photo l117 : -3974581484 Missing photo l117 : -3974581488 Missing photo l117 : -3974581503 Missing photo l117 : -3974581516 Missing photo l117 : -3974581526 Missing photo l117 : -3974581528 Missing photo l117 : -3974581543 Missing photo l117 : -3974581554 Missing photo l117 : -3974581555 Missing photo l117 : -3974581583 To insert : -3974581457 Missing photo l134 : -3974581457 To insert : -3974581482 Missing photo l134 : -3974581482 To insert : -3974581489 Missing photo l134 : -3974581489 To insert : -3974581500 Missing photo l134 : -3974581500 To insert : -3974581507 Missing photo l134 : -3974581507 To insert : -3974581551 Missing photo l134 : -3974581551 To insert : -3974581553 Missing photo l134 : -3974581553 To insert : -3974581577 Missing photo l134 : -3974581577 To insert : -3974581465 Missing photo l134 : -3974581465 To insert : -3974581473 Missing photo l134 : -3974581473 To insert : -3974581480 Missing photo l134 : -3974581480 To insert : -3974581502 Missing photo l134 : -3974581502 To insert : -3974581504 Missing photo l134 : -3974581504 To insert : -3974581501 Missing photo l134 : -3974581501 To insert : -3974581499 Missing photo l134 : -3974581499 To insert : -3974581514 Missing photo l134 : -3974581514 To insert : -3974581508 Missing photo l134 : -3974581508 To insert : -3974581527 Missing photo l134 : -3974581527 To insert : -3974581550 Missing photo l134 : -3974581550 To insert : -3974581571 Missing photo l134 : -3974581571 To insert : -3974581564 Missing photo l134 : -3974581564 To insert : -3974581486 Missing photo l134 : -3974581486 To insert : -3974581541 Missing photo l134 : -3974581541 To insert : -3974581567 Missing photo l134 : -3974581567 To insert : -3974581563 Missing photo l134 : -3974581563 To insert : -3974581460 Missing photo l134 : -3974581460 To insert : -3974581459 Missing photo l134 : -3974581459 To insert : -3974581481 Missing photo l134 : -3974581481 To insert : -3974581478 Missing photo l134 : -3974581478 To insert : -3974581472 Missing photo l134 : -3974581472 To insert : -3974581487 Missing photo l134 : -3974581487 To insert : -3974581505 Missing photo l134 : -3974581505 To insert : -3974581515 Missing photo l134 : -3974581515 To insert : -3974581512 Missing photo l134 : -3974581512 To insert : -3974581506 Missing photo l134 : -3974581506 To insert : -3974581520 Missing photo l134 : -3974581520 To insert : -3974581519 Missing photo l134 : -3974581519 To insert : -3974581525 Missing photo l134 : -3974581525 To insert : -3974581531 Missing photo l134 : -3974581531 To insert : -3974581533 Missing photo l134 : -3974581533 To insert : -3974581535 Missing photo l134 : -3974581535 To insert : -3976615276 Missing photo l134 : -3976615276 To insert : -3974581547 Missing photo l134 : -3974581547 To insert : -3974581548 Missing photo l134 : -3974581548 To insert : -3974581552 Missing photo l134 : -3974581552 To insert : -3974581558 Missing photo l134 : -3974581558 To insert : -3974581566 Missing photo l134 : -3974581566 To insert : -3974581569 Missing photo l134 : -3974581569 To insert : -3974581578 Missing photo l134 : -3974581578 To insert : -3974581580 Missing photo l134 : -3974581580 To insert : -3974581573 Missing photo l134 : -3974581573 To insert : -3974581454 Missing photo l134 : -3974581454 To insert : -3974581455 Missing photo l134 : -3974581455 To insert : -3974581458 Missing photo l134 : -3974581458 To insert : -3974581452 Missing photo l134 : -3974581452 To insert : -3974581453 Missing photo l134 : -3974581453 To insert : -3974581467 Missing photo l134 : -3974581467 To insert : -3974581471 Missing photo l134 : -3974581471 To insert : -3974581469 Missing photo l134 : -3974581469 To insert : -3974581470 Missing photo l134 : -3974581470 To insert : -3974581475 Missing photo l134 : -3974581475 To insert : -3974581468 Missing photo l134 : -3974581468 To insert : -3974581485 Missing photo l134 : -3974581485 To insert : -3974581483 Missing photo l134 : -3974581483 To insert : -3974581497 Missing photo l134 : -3974581497 To insert : -3974581495 Missing photo l134 : -3974581495 To insert : -3974581498 Missing photo l134 : -3974581498 To insert : -3976615275 Missing photo l134 : -3976615275 To insert : -3974581509 Missing photo l134 : -3974581509 To insert : -3974581511 Missing photo l134 : -3974581511 To insert : -3974581521 Missing photo l134 : -3974581521 To insert : -3974581518 Missing photo l134 : -3974581518 To insert : -3974581545 Missing photo l134 : -3974581545 To insert : -3974581565 Missing photo l134 : -3974581565 To insert : -3974581559 Missing photo l134 : -3974581559 To insert : -3974581561 Missing photo l134 : -3974581561 To insert : -3974581560 Missing photo l134 : -3974581560 To insert : -3974581576 Missing photo l134 : -3974581576 To insert : -3974581572 Missing photo l134 : -3974581572 To insert : -3974581575 Missing photo l134 : -3974581575 To insert : -3974581574 Missing photo l134 : -3974581574 To insert : -3976615278 Missing photo l134 : -3976615278 To insert : -3974581456 Missing photo l134 : -3974581456 To insert : -3974581476 Missing photo l134 : -3974581476 To insert : -3974581479 Missing photo l134 : -3974581479 To insert : -3974581484 Missing photo l134 : -3974581484 To insert : -3974581488 Missing photo l134 : -3974581488 To insert : -3974581503 Missing photo l134 : -3974581503 To insert : -3974581516 Missing photo l134 : -3974581516 To insert : -3974581526 Missing photo l134 : -3974581526 To insert : -3974581528 Missing photo l134 : -3974581528 To insert : -3974581543 Missing photo l134 : -3974581543 To insert : -3974581554 Missing photo l134 : -3974581554 To insert : -3974581555 Missing photo l134 : -3974581555 To insert : -3974581583 Missing photo l134 : -3974581583 time to insert the descriptors : 20.616047143936157 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : tfhub_classification2 we use saveGeneral [1386056377, 1386056344, 1386056336, 1386056328, 1386056324, 1386055709, 1386055705, 1386055687, 1386055658, 1386055654, 1386055621] Looping around the photos to save general results len do output : 95 /-3974581457Didn't retrieve data . /-3974581482Didn't retrieve data . /-3974581489Didn't retrieve data . /-3974581500Didn't retrieve data . /-3974581507Didn't retrieve data . /-3974581551Didn't retrieve data . /-3974581553Didn't retrieve data . /-3974581577Didn't retrieve data . /-3974581465Didn't retrieve data . /-3974581473Didn't retrieve data . /-3974581480Didn't retrieve data . /-3974581502Didn't retrieve data . /-3974581504Didn't retrieve data . /-3974581501Didn't retrieve data . /-3974581499Didn't retrieve data . /-3974581514Didn't retrieve data . /-3974581508Didn't retrieve data . /-3974581527Didn't retrieve data . /-3974581550Didn't retrieve data . /-3974581571Didn't retrieve data . /-3974581564Didn't retrieve data . /-3974581486Didn't retrieve data . /-3974581541Didn't retrieve data . /-3974581567Didn't retrieve data . /-3974581563Didn't retrieve data . /-3974581460Didn't retrieve data . /-3974581459Didn't retrieve data . /-3974581481Didn't retrieve data . /-3974581478Didn't retrieve data . /-3974581472Didn't retrieve data . /-3974581487Didn't retrieve data . /-3974581505Didn't retrieve data . /-3974581515Didn't retrieve data . /-3974581512Didn't retrieve data . /-3974581506Didn't retrieve data . /-3974581520Didn't retrieve data . /-3974581519Didn't retrieve data . /-3974581525Didn't retrieve data . /-3974581531Didn't retrieve data . /-3974581533Didn't retrieve data . /-3974581535Didn't retrieve data . /-3976615276Didn't retrieve data . /-3974581547Didn't retrieve data . /-3974581548Didn't retrieve data . /-3974581552Didn't retrieve data . /-3974581558Didn't retrieve data . /-3974581566Didn't retrieve data . /-3974581569Didn't retrieve data . /-3974581578Didn't retrieve data . /-3974581580Didn't retrieve data . /-3974581573Didn't retrieve data . /-3974581454Didn't retrieve data . /-3974581455Didn't retrieve data . /-3974581458Didn't retrieve data . /-3974581452Didn't retrieve data . /-3974581453Didn't retrieve data . /-3974581467Didn't retrieve data . /-3974581471Didn't retrieve data . /-3974581469Didn't retrieve data . /-3974581470Didn't retrieve data . /-3974581475Didn't retrieve data . /-3974581468Didn't retrieve data . /-3974581485Didn't retrieve data . /-3974581483Didn't retrieve data . /-3974581497Didn't retrieve data . /-3974581495Didn't retrieve data . /-3974581498Didn't retrieve data . /-3976615275Didn't retrieve data . /-3974581509Didn't retrieve data . /-3974581511Didn't retrieve data . /-3974581521Didn't retrieve data . /-3974581518Didn't retrieve data . /-3974581545Didn't retrieve data . /-3974581565Didn't retrieve data . /-3974581559Didn't retrieve data . /-3974581561Didn't retrieve data . /-3974581560Didn't retrieve data . /-3974581576Didn't retrieve data . /-3974581572Didn't retrieve data . /-3974581575Didn't retrieve data . /-3974581574Didn't retrieve data . /-3976615278Didn't retrieve data . /-3974581456Didn't retrieve data . /-3974581476Didn't retrieve data . /-3974581479Didn't retrieve data . /-3974581484Didn't retrieve data . /-3974581488Didn't retrieve data . /-3974581503Didn't retrieve data . /-3974581516Didn't retrieve data . /-3974581526Didn't retrieve data . /-3974581528Didn't retrieve data . /-3974581543Didn't retrieve data . /-3974581554Didn't retrieve data . /-3974581555Didn't retrieve data . /-3974581583Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056377', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056344', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056336', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056328', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056324', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055709', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055705', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055687', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055658', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055654', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055621', None, None, None, None, None, '3777389') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 106 time used for this insertion : 0.028033018112182617 save_final save missing photos in datou_result : time spend for datou_step_exec : 54.24276161193848 time spend to save output : 0.03056645393371582 total time spend for step 6 : 54.27332806587219 step7:merge_mask_thcl_custom Fri Sep 26 11:05:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 141 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1386056377, 1386056344, 1386056336, 1386056328, 1386056324, 1386055709, 1386055705, 1386055687, 1386055658, 1386055654, 1386055621] Looping around the photos to save general results len do output : 11 /1386056377Didn't retrieve data .Didn't retrieve data . /1386056344Didn't retrieve data .Didn't retrieve data . /1386056336Didn't retrieve data .Didn't retrieve data . /1386056328Didn't retrieve data .Didn't retrieve data . /1386056324Didn't retrieve data .Didn't retrieve data . /1386055709Didn't retrieve data .Didn't retrieve data . /1386055705Didn't retrieve data .Didn't retrieve data . /1386055687Didn't retrieve data .Didn't retrieve data . /1386055658Didn't retrieve data .Didn't retrieve data . /1386055654Didn't retrieve data .Didn't retrieve data . /1386055621Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056377', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056344', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056336', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056328', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056324', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055709', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055705', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055687', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055658', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055654', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055621', None, None, None, None, None, '3777389') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 33 time used for this insertion : 0.013291358947753906 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.06896758079528809 time spend to save output : 0.013818979263305664 total time spend for step 7 : 0.08278656005859375 step8:rle_unique_nms_with_priority Fri Sep 26 11:05:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 11 nb_hashtags : 4 time to prepare the origin masks : 10.84582781791687 time for calcul the mask position with numpy : 0.7122559547424316 nb_pixel_total : 8537974 time to create 1 rle with new method : 1.5546379089355469 time for calcul the mask position with numpy : 0.05821537971496582 nb_pixel_total : 82931 time to create 1 rle with old method : 0.09310269355773926 time for calcul the mask position with numpy : 0.08094978332519531 nb_pixel_total : 953653 time to create 1 rle with new method : 0.5997073650360107 time for calcul the mask position with numpy : 0.05498385429382324 nb_pixel_total : 5181 time to create 1 rle with old method : 0.005951642990112305 time for calcul the mask position with numpy : 0.06355166435241699 nb_pixel_total : 702440 time to create 1 rle with new method : 0.5544440746307373 time for calcul the mask position with numpy : 0.059775590896606445 nb_pixel_total : 1379700 time to create 1 rle with new method : 0.5840325355529785 time for calcul the mask position with numpy : 0.05078434944152832 nb_pixel_total : 149269 time to create 1 rle with old method : 0.16416120529174805 time for calcul the mask position with numpy : 0.03482198715209961 nb_pixel_total : 16258 time to create 1 rle with old method : 0.0182797908782959 time for calcul the mask position with numpy : 0.035331010818481445 nb_pixel_total : 25522 time to create 1 rle with old method : 0.0286257266998291 time for calcul the mask position with numpy : 0.04015803337097168 nb_pixel_total : 410163 time to create 1 rle with new method : 0.6675584316253662 time for calcul the mask position with numpy : 0.036773681640625 nb_pixel_total : 40570 time to create 1 rle with old method : 0.04542875289916992 time for calcul the mask position with numpy : 0.03788137435913086 nb_pixel_total : 26579 time to create 1 rle with old method : 0.04289507865905762 create new chi : 5.826998710632324 time to delete rle : 0.017018795013427734 batch 1 Loaded 13 chid ids of type : 3726 Number RLEs to save : 18998 TO DO : save crop sub photo not yet done ! save time : 1.1685011386871338 nb_obj : 16 nb_hashtags : 4 time to prepare the origin masks : 19.66213607788086 time for calcul the mask position with numpy : 0.5006675720214844 nb_pixel_total : 7737904 time to create 1 rle with new method : 1.073911428451538 time for calcul the mask position with numpy : 0.10082054138183594 nb_pixel_total : 308386 time to create 1 rle with new method : 1.0062761306762695 time for calcul the mask position with numpy : 0.08897781372070312 nb_pixel_total : 20897 time to create 1 rle with old method : 0.02464604377746582 time for calcul the mask position with numpy : 0.07811379432678223 nb_pixel_total : 7399 time to create 1 rle with old method : 0.00872659683227539 time for calcul the mask position with numpy : 0.05039834976196289 nb_pixel_total : 27929 time to create 1 rle with old method : 0.033441781997680664 time for calcul the mask position with numpy : 0.04961681365966797 nb_pixel_total : 33927 time to create 1 rle with old method : 0.03753471374511719 time for calcul the mask position with numpy : 0.055957794189453125 nb_pixel_total : 777582 time to create 1 rle with new method : 1.503209114074707 time for calcul the mask position with numpy : 0.04702186584472656 nb_pixel_total : 276785 time to create 1 rle with new method : 0.9653210639953613 time for calcul the mask position with numpy : 0.0717475414276123 nb_pixel_total : 135425 time to create 1 rle with old method : 0.1406879425048828 time for calcul the mask position with numpy : 0.040624141693115234 nb_pixel_total : 29881 time to create 1 rle with old method : 0.031059741973876953 time for calcul the mask position with numpy : 0.04539036750793457 nb_pixel_total : 4734 time to create 1 rle with old method : 0.006768703460693359 time for calcul the mask position with numpy : 0.06062459945678711 nb_pixel_total : 1245442 time to create 1 rle with new method : 0.9212021827697754 time for calcul the mask position with numpy : 0.048410892486572266 nb_pixel_total : 292659 time to create 1 rle with new method : 1.1068973541259766 time for calcul the mask position with numpy : 0.04954838752746582 nb_pixel_total : 97263 time to create 1 rle with old method : 0.10665559768676758 time for calcul the mask position with numpy : 0.06490707397460938 nb_pixel_total : 219191 time to create 1 rle with new method : 0.7269449234008789 time for calcul the mask position with numpy : 0.0711057186126709 nb_pixel_total : 17442 time to create 1 rle with old method : 0.019734859466552734 time for calcul the mask position with numpy : 0.07945823669433594 nb_pixel_total : 1097394 time to create 1 rle with new method : 0.9959092140197754 create new chi : 10.63280701637268 time to delete rle : 0.0027379989624023438 batch 1 Loaded 18 chid ids of type : 3726 Number RLEs to save : 26031 TO DO : save crop sub photo not yet done ! save time : 1.5644726753234863 nb_obj : 9 nb_hashtags : 4 time to prepare the origin masks : 14.257596969604492 time for calcul the mask position with numpy : 0.8357689380645752 nb_pixel_total : 11236261 time to create 1 rle with new method : 0.9149069786071777 time for calcul the mask position with numpy : 0.0874333381652832 nb_pixel_total : 61563 time to create 1 rle with old method : 0.07608342170715332 time for calcul the mask position with numpy : 0.06804466247558594 nb_pixel_total : 206912 time to create 1 rle with new method : 1.02079439163208 time for calcul the mask position with numpy : 0.06171989440917969 nb_pixel_total : 4767 time to create 1 rle with old method : 0.005504608154296875 time for calcul the mask position with numpy : 0.06809234619140625 nb_pixel_total : 410339 time to create 1 rle with new method : 0.997290849685669 time for calcul the mask position with numpy : 0.0669708251953125 nb_pixel_total : 103922 time to create 1 rle with old method : 0.12604165077209473 time for calcul the mask position with numpy : 0.06604909896850586 nb_pixel_total : 55130 time to create 1 rle with old method : 0.059392452239990234 time for calcul the mask position with numpy : 0.061704158782958984 nb_pixel_total : 6130 time to create 1 rle with old method : 0.006700277328491211 time for calcul the mask position with numpy : 0.07539820671081543 nb_pixel_total : 148512 time to create 1 rle with old method : 0.1584630012512207 time for calcul the mask position with numpy : 0.06956672668457031 nb_pixel_total : 96704 time to create 1 rle with old method : 0.1125485897064209 create new chi : 5.093808650970459 time to delete rle : 0.0013816356658935547 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 10061 TO DO : save crop sub photo not yet done ! save time : 0.6290781497955322 nb_obj : 12 nb_hashtags : 3 time to prepare the origin masks : 14.714598655700684 time for calcul the mask position with numpy : 0.5375592708587646 nb_pixel_total : 9676085 time to create 1 rle with new method : 0.6803665161132812 time for calcul the mask position with numpy : 0.040251731872558594 nb_pixel_total : 25074 time to create 1 rle with old method : 0.0666048526763916 time for calcul the mask position with numpy : 0.043253421783447266 nb_pixel_total : 14782 time to create 1 rle with old method : 0.018592119216918945 time for calcul the mask position with numpy : 0.048897743225097656 nb_pixel_total : 54553 time to create 1 rle with old method : 0.060993194580078125 time for calcul the mask position with numpy : 0.039927005767822266 nb_pixel_total : 229705 time to create 1 rle with new method : 0.7943391799926758 time for calcul the mask position with numpy : 0.041220903396606445 nb_pixel_total : 70714 time to create 1 rle with old method : 0.07346224784851074 time for calcul the mask position with numpy : 0.03731179237365723 nb_pixel_total : 148777 time to create 1 rle with old method : 0.15874075889587402 time for calcul the mask position with numpy : 0.06440281867980957 nb_pixel_total : 24019 time to create 1 rle with old method : 0.0253450870513916 time for calcul the mask position with numpy : 0.059433937072753906 nb_pixel_total : 24560 time to create 1 rle with old method : 0.02616119384765625 time for calcul the mask position with numpy : 0.05709505081176758 nb_pixel_total : 14126 time to create 1 rle with old method : 0.01898336410522461 time for calcul the mask position with numpy : 0.08069992065429688 nb_pixel_total : 96321 time to create 1 rle with old method : 0.10771846771240234 time for calcul the mask position with numpy : 0.059449195861816406 nb_pixel_total : 17196 time to create 1 rle with old method : 0.01850748062133789 time for calcul the mask position with numpy : 0.07554101943969727 nb_pixel_total : 1934328 time to create 1 rle with new method : 0.8994085788726807 create new chi : 4.267426490783691 time to delete rle : 0.0016138553619384766 batch 1 Loaded 13 chid ids of type : 3726 Number RLEs to save : 16168 TO DO : save crop sub photo not yet done ! save time : 0.9780867099761963 nb_obj : 12 nb_hashtags : 5 time to prepare the origin masks : 17.08155870437622 time for calcul the mask position with numpy : 0.3930394649505615 nb_pixel_total : 7759761 time to create 1 rle with new method : 0.6267397403717041 time for calcul the mask position with numpy : 0.03805351257324219 nb_pixel_total : 73052 time to create 1 rle with old method : 0.09891653060913086 time for calcul the mask position with numpy : 0.05192136764526367 nb_pixel_total : 440766 time to create 1 rle with new method : 2.3050343990325928 time for calcul the mask position with numpy : 0.048774003982543945 nb_pixel_total : 341056 time to create 1 rle with new method : 1.4406278133392334 time for calcul the mask position with numpy : 0.0606997013092041 nb_pixel_total : 4604 time to create 1 rle with old method : 0.005027294158935547 time for calcul the mask position with numpy : 0.09088659286499023 nb_pixel_total : 1398650 time to create 1 rle with new method : 0.8306500911712646 time for calcul the mask position with numpy : 0.06139183044433594 nb_pixel_total : 44087 time to create 1 rle with old method : 0.05020499229431152 time for calcul the mask position with numpy : 0.0628514289855957 nb_pixel_total : 729532 time to create 1 rle with new method : 0.90852952003479 time for calcul the mask position with numpy : 0.046439170837402344 nb_pixel_total : 33604 time to create 1 rle with old method : 0.03768014907836914 time for calcul the mask position with numpy : 0.058750152587890625 nb_pixel_total : 22971 time to create 1 rle with old method : 0.026644468307495117 time for calcul the mask position with numpy : 0.05002546310424805 nb_pixel_total : 381388 time to create 1 rle with new method : 0.7564098834991455 time for calcul the mask position with numpy : 0.047931671142578125 nb_pixel_total : 299614 time to create 1 rle with new method : 0.9894497394561768 time for calcul the mask position with numpy : 0.057702064514160156 nb_pixel_total : 801155 time to create 1 rle with new method : 1.1205875873565674 create new chi : 10.638052463531494 time to delete rle : 0.0030035972595214844 batch 1 Loaded 14 chid ids of type : 3726 Number RLEs to save : 22432 TO DO : save crop sub photo not yet done ! save time : 1.346853256225586 nb_obj : 11 nb_hashtags : 4 time to prepare the origin masks : 15.22331166267395 time for calcul the mask position with numpy : 0.5739445686340332 nb_pixel_total : 10822118 time to create 1 rle with new method : 0.5819520950317383 time for calcul the mask position with numpy : 0.035688161849975586 nb_pixel_total : 28108 time to create 1 rle with old method : 0.032529354095458984 time for calcul the mask position with numpy : 0.03580617904663086 nb_pixel_total : 124480 time to create 1 rle with old method : 0.1403343677520752 time for calcul the mask position with numpy : 0.06032443046569824 nb_pixel_total : 24682 time to create 1 rle with old method : 0.02706766128540039 time for calcul the mask position with numpy : 0.06008172035217285 nb_pixel_total : 192544 time to create 1 rle with new method : 0.7399568557739258 time for calcul the mask position with numpy : 0.05896592140197754 nb_pixel_total : 190953 time to create 1 rle with new method : 0.8234519958496094 time for calcul the mask position with numpy : 0.06204509735107422 nb_pixel_total : 11134 time to create 1 rle with old method : 0.012974739074707031 time for calcul the mask position with numpy : 0.06564974784851074 nb_pixel_total : 429915 time to create 1 rle with new method : 0.8000152111053467 time for calcul the mask position with numpy : 0.05896782875061035 nb_pixel_total : 6554 time to create 1 rle with old method : 0.007821083068847656 time for calcul the mask position with numpy : 0.06206059455871582 nb_pixel_total : 83417 time to create 1 rle with old method : 0.09169936180114746 time for calcul the mask position with numpy : 0.05369377136230469 nb_pixel_total : 5628 time to create 1 rle with old method : 0.0074312686920166016 time for calcul the mask position with numpy : 0.03721761703491211 nb_pixel_total : 410707 time to create 1 rle with new method : 0.6425485610961914 create new chi : 5.290550708770752 time to delete rle : 0.0015680789947509766 batch 1 Loaded 12 chid ids of type : 3726 Number RLEs to save : 13440 TO DO : save crop sub photo not yet done ! save time : 0.7589967250823975 nb_obj : 7 nb_hashtags : 3 time to prepare the origin masks : 7.832942962646484 time for calcul the mask position with numpy : 1.4827749729156494 nb_pixel_total : 10052580 time to create 1 rle with new method : 1.543947458267212 time for calcul the mask position with numpy : 0.0678400993347168 nb_pixel_total : 147601 time to create 1 rle with old method : 0.17733097076416016 time for calcul the mask position with numpy : 0.05833888053894043 nb_pixel_total : 32128 time to create 1 rle with old method : 0.044260501861572266 time for calcul the mask position with numpy : 0.06260108947753906 nb_pixel_total : 247527 time to create 1 rle with new method : 1.1659789085388184 time for calcul the mask position with numpy : 0.04400062561035156 nb_pixel_total : 112401 time to create 1 rle with old method : 0.12561392784118652 time for calcul the mask position with numpy : 0.039847373962402344 nb_pixel_total : 57034 time to create 1 rle with old method : 0.06406235694885254 time for calcul the mask position with numpy : 0.041062355041503906 nb_pixel_total : 65423 time to create 1 rle with old method : 0.07280755043029785 time for calcul the mask position with numpy : 0.08107495307922363 nb_pixel_total : 1615546 time to create 1 rle with new method : 1.1738531589508057 create new chi : 6.408381938934326 time to delete rle : 0.0013322830200195312 batch 1 Loaded 9 chid ids of type : 3726 Number RLEs to save : 12300 TO DO : save crop sub photo not yet done ! save time : 0.7430596351623535 nb_obj : 5 nb_hashtags : 3 time to prepare the origin masks : 7.4695210456848145 time for calcul the mask position with numpy : 0.841158390045166 nb_pixel_total : 11744833 time to create 1 rle with new method : 1.0633344650268555 time for calcul the mask position with numpy : 0.03976035118103027 nb_pixel_total : 267163 time to create 1 rle with new method : 0.9180774688720703 time for calcul the mask position with numpy : 0.03801727294921875 nb_pixel_total : 15718 time to create 1 rle with old method : 0.018036365509033203 time for calcul the mask position with numpy : 0.037927865982055664 nb_pixel_total : 39427 time to create 1 rle with old method : 0.04457497596740723 time for calcul the mask position with numpy : 0.042026519775390625 nb_pixel_total : 196896 time to create 1 rle with new method : 0.9579861164093018 time for calcul the mask position with numpy : 0.06561160087585449 nb_pixel_total : 66203 time to create 1 rle with old method : 0.07632279396057129 create new chi : 4.297360420227051 time to delete rle : 0.0012669563293457031 batch 1 Loaded 6 chid ids of type : 3726 Number RLEs to save : 6647 TO DO : save crop sub photo not yet done ! save time : 0.3990194797515869 nb_obj : 6 nb_hashtags : 3 time to prepare the origin masks : 8.841273784637451 time for calcul the mask position with numpy : 1.032149314880371 nb_pixel_total : 11651791 time to create 1 rle with new method : 1.179692029953003 time for calcul the mask position with numpy : 0.06443357467651367 nb_pixel_total : 125012 time to create 1 rle with old method : 0.1367945671081543 time for calcul the mask position with numpy : 0.06331682205200195 nb_pixel_total : 25115 time to create 1 rle with old method : 0.027698278427124023 time for calcul the mask position with numpy : 0.061333417892456055 nb_pixel_total : 8548 time to create 1 rle with old method : 0.009212732315063477 time for calcul the mask position with numpy : 0.0622401237487793 nb_pixel_total : 22618 time to create 1 rle with old method : 0.02503180503845215 time for calcul the mask position with numpy : 0.04394340515136719 nb_pixel_total : 465920 time to create 1 rle with new method : 0.9276161193847656 time for calcul the mask position with numpy : 0.04727458953857422 nb_pixel_total : 31236 time to create 1 rle with old method : 0.03528165817260742 create new chi : 3.822192907333374 time to delete rle : 0.0019371509552001953 batch 1 Loaded 7 chid ids of type : 3726 Number RLEs to save : 6770 TO DO : save crop sub photo not yet done ! save time : 0.41747546195983887 nb_obj : 11 nb_hashtags : 4 time to prepare the origin masks : 12.73478364944458 time for calcul the mask position with numpy : 0.4246528148651123 nb_pixel_total : 10167527 time to create 1 rle with new method : 1.2228474617004395 time for calcul the mask position with numpy : 0.04604601860046387 nb_pixel_total : 23455 time to create 1 rle with old method : 0.029043197631835938 time for calcul the mask position with numpy : 0.04738593101501465 nb_pixel_total : 389333 time to create 1 rle with new method : 1.0346412658691406 time for calcul the mask position with numpy : 0.07179045677185059 nb_pixel_total : 7952 time to create 1 rle with old method : 0.009829998016357422 time for calcul the mask position with numpy : 0.0694127082824707 nb_pixel_total : 101438 time to create 1 rle with old method : 0.11878180503845215 time for calcul the mask position with numpy : 0.08318805694580078 nb_pixel_total : 85410 time to create 1 rle with old method : 0.09585881233215332 time for calcul the mask position with numpy : 0.06471705436706543 nb_pixel_total : 36719 time to create 1 rle with old method : 0.04092860221862793 time for calcul the mask position with numpy : 0.06354093551635742 nb_pixel_total : 136618 time to create 1 rle with old method : 0.15158629417419434 time for calcul the mask position with numpy : 0.0623934268951416 nb_pixel_total : 15595 time to create 1 rle with old method : 0.018146038055419922 time for calcul the mask position with numpy : 0.07743263244628906 nb_pixel_total : 185212 time to create 1 rle with new method : 0.9857418537139893 time for calcul the mask position with numpy : 0.05163168907165527 nb_pixel_total : 115375 time to create 1 rle with old method : 0.1339716911315918 time for calcul the mask position with numpy : 0.07600951194763184 nb_pixel_total : 1065606 time to create 1 rle with new method : 1.016420841217041 create new chi : 6.211317777633667 time to delete rle : 0.0035719871520996094 batch 1 Loaded 12 chid ids of type : 3726 Number RLEs to save : 16178 TO DO : save crop sub photo not yet done ! save time : 1.0203278064727783 nb_obj : 13 nb_hashtags : 4 time to prepare the origin masks : 15.835212230682373 time for calcul the mask position with numpy : 0.5424137115478516 nb_pixel_total : 9930417 time to create 1 rle with new method : 1.3203046321868896 time for calcul the mask position with numpy : 0.07874464988708496 nb_pixel_total : 28042 time to create 1 rle with old method : 0.04113459587097168 time for calcul the mask position with numpy : 0.0776224136352539 nb_pixel_total : 550 time to create 1 rle with old method : 0.0012657642364501953 time for calcul the mask position with numpy : 0.08954715728759766 nb_pixel_total : 46948 time to create 1 rle with old method : 0.05727124214172363 time for calcul the mask position with numpy : 0.08214879035949707 nb_pixel_total : 35195 time to create 1 rle with old method : 0.04273509979248047 time for calcul the mask position with numpy : 0.07445955276489258 nb_pixel_total : 1134012 time to create 1 rle with new method : 1.111513376235962 time for calcul the mask position with numpy : 0.05273890495300293 nb_pixel_total : 161241 time to create 1 rle with new method : 1.0374817848205566 time for calcul the mask position with numpy : 0.06439042091369629 nb_pixel_total : 23751 time to create 1 rle with old method : 0.025052309036254883 time for calcul the mask position with numpy : 0.07036852836608887 nb_pixel_total : 15353 time to create 1 rle with old method : 0.016381025314331055 time for calcul the mask position with numpy : 0.07222127914428711 nb_pixel_total : 94403 time to create 1 rle with old method : 0.09900975227355957 time for calcul the mask position with numpy : 0.07394289970397949 nb_pixel_total : 270351 time to create 1 rle with new method : 1.0891406536102295 time for calcul the mask position with numpy : 0.07959985733032227 nb_pixel_total : 104426 time to create 1 rle with old method : 0.11353564262390137 time for calcul the mask position with numpy : 0.08152556419372559 nb_pixel_total : 31256 time to create 1 rle with old method : 0.03279757499694824 time for calcul the mask position with numpy : 0.05900716781616211 nb_pixel_total : 454295 time to create 1 rle with new method : 0.9946637153625488 create new chi : 7.7544105052948 time to delete rle : 0.0023860931396484375 batch 1 Loaded 17 chid ids of type : 3726 Number RLEs to save : 16133 TO DO : save crop sub photo not yet done ! save time : 0.9760692119598389 map_output_result : {1386056377: (0.0, 'Should be the crop_list due to order', 0.0), 1386056344: (0.0, 'Should be the crop_list due to order', 0.0), 1386056336: (0.0, 'Should be the crop_list due to order', 0.0), 1386056328: (0.0, 'Should be the crop_list due to order', 0.0), 1386056324: (0.0, 'Should be the crop_list due to order', 0.0), 1386055709: (0.0, 'Should be the crop_list due to order', 0.0), 1386055705: (0.0, 'Should be the crop_list due to order', 0.0), 1386055687: (0.0, 'Should be the crop_list due to order', 0.0), 1386055658: (0.0, 'Should be the crop_list due to order', 0.0), 1386055654: (0.0, 'Should be the crop_list due to order', 0.0), 1386055621: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1386056377, 1386056344, 1386056336, 1386056328, 1386056324, 1386055709, 1386055705, 1386055687, 1386055658, 1386055654, 1386055621] Looping around the photos to save general results len do output : 11 /1386056377.Didn't retrieve data . /1386056344.Didn't retrieve data . /1386056336.Didn't retrieve data . /1386056328.Didn't retrieve data . /1386056324.Didn't retrieve data . /1386055709.Didn't retrieve data . /1386055705.Didn't retrieve data . /1386055687.Didn't retrieve data . /1386055658.Didn't retrieve data . /1386055654.Didn't retrieve data . /1386055621.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056377', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056344', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056336', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056328', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056324', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055709', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055705', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055687', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055658', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055654', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055621', None, None, None, None, None, '3777389') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 33 time used for this insertion : 0.019452810287475586 save_final save missing photos in datou_result : time spend for datou_step_exec : 228.8320825099945 time spend to save output : 0.12450242042541504 total time spend for step 8 : 228.95658493041992 step9:crop_condition Fri Sep 26 11:09:19 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 11 ! batch 1 Loaded 131 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! About to upload 8 photos upload in portfolio : 4869462 init cache_photo without model_param we have 8 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758877772_30252 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 8 photos in the portfolio 4869462 time of upload the photos Elapsed time : 4.497334003448486 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758877781_30252 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.2321925163269043 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 24 About to insert : list_path_to_insert length 24 new photo from crops ! About to upload 24 photos upload in portfolio : 4869462 init cache_photo without model_param we have 24 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758877796_30252 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 24 photos in the portfolio 4869462 time of upload the photos Elapsed time : 7.0158257484436035 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758877809_30252 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first Unexecpected behavior in 07/2025 that can be generalized l287 : type_extension .jpg This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.3092072010040283 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1386056377, 1386056344, 1386056336, 1386056328, 1386056324, 1386055709, 1386055705, 1386055687, 1386055658, 1386055654, 1386055621] Looping around the photos to save general results len do output : 36 /1386327902Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327903Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327904Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327905Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327906Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327907Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327908Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327909Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327914Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327915Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327923Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327924Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327925Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327926Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327927Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327928Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327929Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327930Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327931Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327932Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327933Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327934Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327935Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327936Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327937Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327938Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327939Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327940Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327941Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327942Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327943Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327944Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327945Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327946Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327949Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386327950Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056377', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056344', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056336', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056328', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056324', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055709', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055705', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055687', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055658', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055654', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055621', None, None, None, None, None, '3777389') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 119 time used for this insertion : 0.031771183013916016 save_final save missing photos in datou_result : time spend for datou_step_exec : 51.86248707771301 time spend to save output : 0.03342628479003906 total time spend for step 9 : 51.89591336250305 step10:ventilate_hashtags_in_portfolio Fri Sep 26 11:10:11 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 27224793 get user id for portfolio 27224793 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27224793 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','Carton_gris','cartonnette','Carton_brun','autre_refus','metal','plastique','environnement','flou','Teint_Dans_La_Masse','papier','kraft')) AND mptpi.`min_score`=0.5 To do To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27224793 AND mptpi.`type`=4417 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','Carton_gris','cartonnette','Carton_brun','autre_refus','metal','plastique','environnement','flou','Teint_Dans_La_Masse','papier','kraft')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://marlene.fotonower.com/velours/27224976,27224977,27224978,27224979,27224980,27224981,27224982,27224983,27224984,27224985,27224986,27224987?tags=Carton_gris,Carton_brun,cartonnette,environnement,plastique,autre_refus,kraft,metal,mal_croppe,flou,papier,Teint_Dans_La_Masse Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1386056377, 1386056344, 1386056336, 1386056328, 1386056324, 1386055709, 1386055705, 1386055687, 1386055658, 1386055654, 1386055621] Looping around the photos to save general results len do output : 1 /27224793. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056377', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056344', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056336', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056328', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056324', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055709', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055705', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055687', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055658', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055654', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055621', None, None, None, None, None, '3777389') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 12 time used for this insertion : 0.0173642635345459 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.5610489845275879 time spend to save output : 0.01776409149169922 total time spend for step 10 : 0.5788130760192871 step11:final Fri Sep 26 11:10:11 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1386056377: ('0.17544003842563197',), 1386056344: ('0.17544003842563197',), 1386056336: ('0.17544003842563197',), 1386056328: ('0.17544003842563197',), 1386056324: ('0.17544003842563197',), 1386055709: ('0.17544003842563197',), 1386055705: ('0.17544003842563197',), 1386055687: ('0.17544003842563197',), 1386055658: ('0.17544003842563197',), 1386055654: ('0.17544003842563197',), 1386055621: ('0.17544003842563197',)} new output for save of step final : {1386056377: ('0.17544003842563197',), 1386056344: ('0.17544003842563197',), 1386056336: ('0.17544003842563197',), 1386056328: ('0.17544003842563197',), 1386056324: ('0.17544003842563197',), 1386055709: ('0.17544003842563197',), 1386055705: ('0.17544003842563197',), 1386055687: ('0.17544003842563197',), 1386055658: ('0.17544003842563197',), 1386055654: ('0.17544003842563197',), 1386055621: ('0.17544003842563197',)} [1386056377, 1386056344, 1386056336, 1386056328, 1386056324, 1386055709, 1386055705, 1386055687, 1386055658, 1386055654, 1386055621] Looping around the photos to save general results len do output : 11 /1386056377.Didn't retrieve data . /1386056344.Didn't retrieve data . /1386056336.Didn't retrieve data . /1386056328.Didn't retrieve data . /1386056324.Didn't retrieve data . /1386055709.Didn't retrieve data . /1386055705.Didn't retrieve data . /1386055687.Didn't retrieve data . /1386055658.Didn't retrieve data . /1386055654.Didn't retrieve data . /1386055621.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056377', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056344', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056336', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056328', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056324', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055709', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055705', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055687', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055658', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055654', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055621', None, None, None, None, None, '3777389') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 33 time used for this insertion : 0.015044450759887695 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.14416074752807617 time spend to save output : 0.016582727432250977 total time spend for step 11 : 0.16074347496032715 step12:velours_tree Fri Sep 26 11:10:12 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.3975038528442383 time spend to save output : 3.790855407714844e-05 total time spend for step 12 : 0.39754176139831543 step13:send_mail_cod Fri Sep 26 11:10:12 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin in order to get the selector url, please entre the license of selector results_Auto_P27224793_26-09-2025_11_10_12.pdf 27224966 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272249661758877812 27224968 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272249681758877814 27224969 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272249691758877815 27224970 imagette272249701758877816 27224971 imagette272249711758877816 27224972 imagette272249721758877816 27224973 imagette272249731758877816 27224964 change filename to text .imagette272249641758877816 27224965 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272249651758877817 27224975 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272249751758877819 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=27224793 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://marlene.fotonower.com/velours/27224976,27224977,27224978,27224979,27224980,27224981,27224982,27224983,27224984,27224985,27224986,27224987?tags=Carton_gris,Carton_brun,cartonnette,environnement,plastique,autre_refus,kraft,metal,mal_croppe,flou,papier,Teint_Dans_La_Masse your option no_mail is active, we will not send the real mail to your client args[1386056377] : ((1386056377, -0.010048349306233337, 2107752395), (1386056377, -0.08656857999806743, 492688767), '0.17544003842563197') We are sending mail with results at report@fotonower.com args[1386056344] : ((1386056344, -0.1022894021039678, 496442774), (1386056344, -0.49190306297993985, 492688767), '0.17544003842563197') We are sending mail with results at report@fotonower.com args[1386056336] : ((1386056336, -0.16334335377256753, 496442774), (1386056336, 0.39813007186064836, 492688767), '0.17544003842563197') We are sending mail with results at report@fotonower.com args[1386056328] : ((1386056328, -0.271592460356131, 496442774), (1386056328, -0.24821719520341792, 492688767), '0.17544003842563197') We are sending mail with results at report@fotonower.com args[1386056324] : ((1386056324, -0.485282353416744, 501862349), (1386056324, -0.006098183312578244, 492688767), '0.17544003842563197') We are sending mail with results at report@fotonower.com args[1386055709] : ((1386055709, -0.27502066813998777, 496442774), (1386055709, 0.08331234405272801, 492688767), '0.17544003842563197') We are sending mail with results at report@fotonower.com args[1386055705] : ((1386055705, -0.21439303868175522, 496442774), (1386055705, 0.2220171861946545, 492688767), '0.17544003842563197') We are sending mail with results at report@fotonower.com args[1386055687] : ((1386055687, 0.08035771782907429, 2107752395), (1386055687, 0.23534802152011544, 492688767), '0.17544003842563197') We are sending mail with results at report@fotonower.com args[1386055658] : ((1386055658, -0.15306812144568038, 496442774), (1386055658, -0.7303460062207845, 492688767), '0.17544003842563197') We are sending mail with results at report@fotonower.com args[1386055654] : ((1386055654, -0.24762086422011872, 496442774), (1386055654, -0.5878462885983584, 492688767), '0.17544003842563197') We are sending mail with results at report@fotonower.com args[1386055621] : ((1386055621, -0.32950498706257925, 496442774), (1386055621, 0.47096544337470947, 492688767), '0.17544003842563197') We are sending mail with results at report@fotonower.com refus_total : 0.17544003842563197 2022-04-13 10:29:59 0 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27224793_26-09-2025_11_10_12.pdf results_Auto_P27224793_26-09-2025_11_10_12.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27224793_26-09-2025_11_10_12.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4433','27224793','results_Auto_P27224793_26-09-2025_11_10_12.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27224793_26-09-2025_11_10_12.pdf','pdf','','1.5','0.17544003842563197') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1386056377, 1386056344, 1386056336, 1386056328, 1386056324, 1386055709, 1386055705, 1386055687, 1386055658, 1386055654, 1386055621] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056377', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056344', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056336', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056328', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056324', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055709', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055705', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055687', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055658', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055654', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055621', None, None, None, None, None, '3777389') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.014319658279418945 save_final save missing photos in datou_result : time spend for datou_step_exec : 15.376513957977295 time spend to save output : 0.014626264572143555 total time spend for step 13 : 15.391140222549438 step14:split_time_score Fri Sep 26 11:10:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('10', 26),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 25092025 27224793 Nombre de photos uploadées : 26 / 23040 (0%) 25092025 27224793 Nombre de photos taguées (types de déchets): 0 / 26 (0%) 25092025 27224793 Nombre de photos taguées (volume) : 0 / 26 (0%) elapsed_time : load_data_split_time_score 1.6689300537109375e-06 elapsed_time : order_list_meta_photo_and_scores 5.0067901611328125e-06 ?????????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.001439809799194336 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.2194666862487793 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.17545295764020574 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27224793_26-09-2025_11_10_12.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27224793 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27224793 AND mptpi.`type`=3726 To do Qualite : 0.10785261637899375 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27226523_25-09-2025_14_17_26.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27226523 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27226523 AND mptpi.`type`=3726 To do Qualite : 0.12768736700258843 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27229813_25-09-2025_15_35_55.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27229813 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27229813 AND mptpi.`type`=3726 To do Qualite : 0.16260414343864446 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27266666_26-09-2025_10_39_18.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27266666 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27266666 AND mptpi.`type`=3726 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'25092025': {'nb_upload': 26, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1386056377, 1386056344, 1386056336, 1386056328, 1386056324, 1386055709, 1386055705, 1386055687, 1386055658, 1386055654, 1386055621] Looping around the photos to save general results len do output : 1 /27224793Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056377', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056344', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056336', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056328', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386056324', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055709', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055705', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055687', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055658', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055654', None, None, None, None, None, '3777389') ('4433', None, None, None, None, None, None, None, '3777389') ('4433', None, '1386055621', None, None, None, None, None, '3777389') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 12 time used for this insertion : 0.013923168182373047 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.7154345512390137 time spend to save output : 0.014136552810668945 total time spend for step 14 : 0.7295711040496826 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 11 set_done_treatment 281.71user 244.29system 10:00.11elapsed 87%CPU (0avgtext+0avgdata 4857132maxresident)k 10353928inputs+426632outputs (269122major+29307016minor)pagefaults 0swaps