python /home/admin/mtr/script_for_cron.py -j default -m 20 -a 'python3 ~/workarea/git/Velours/python/prod/datou.py -j batch_current -a 4433' -s traitement_sts -M 0 -S 0 -U 100,80,95 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/home/admin/workarea/git/apy', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 4185575 load datou : 4433 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4433, datou_cur_ids : ['3784106'] with mtr_portfolio_ids : ['27267728'] and first list_photo_ids : [] new path : /proc/4185575/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, brightness, blur_detection, crop_condition, thcl, tfhub_classification2, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 15 list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 15 ; length of list_pids : 15 ; length of list_args : 15 time to download the photos : 4.271055221557617 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 14 step1:mask_detect Fri Sep 26 10:29:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10582 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-09-26 10:29:31.817819: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-26 10:29:31.852631: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-26 10:29:31.854127: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f5d98000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-26 10:29:31.854179: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-26 10:29:31.858754: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-26 10:29:32.056021: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x468f030 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-26 10:29:32.056106: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-26 10:29:32.058418: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 10:29:32.060119: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 10:29:32.091334: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 10:29:32.110422: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 10:29:32.114309: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 10:29:32.146059: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 10:29:32.150999: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 10:29:32.206451: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 10:29:32.208299: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 10:29:32.208777: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 10:29:32.210285: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-26 10:29:32.210305: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-26 10:29:32.210316: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-26 10:29:32.212661: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9801 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-09-26 10:29:32.606925: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 10:29:32.607040: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 10:29:32.607058: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 10:29:32.607074: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 10:29:32.607089: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 10:29:32.607104: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 10:29:32.607119: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 10:29:32.607135: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 10:29:32.608358: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 10:29:32.609538: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 10:29:32.609570: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 10:29:32.609585: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 10:29:32.609604: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 10:29:32.609618: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 10:29:32.609632: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 10:29:32.609645: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 10:29:32.609660: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 10:29:32.610890: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 10:29:32.610934: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-26 10:29:32.610942: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-26 10:29:32.610950: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-26 10:29:32.612231: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9801 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-09-26 10:29:46.507453: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 10:29:46.857157: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 15 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 21 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 3.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 21 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 3.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 10 Detection mask done ! Trying to reset tf kernel 4185769 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5290 tf kernel not reseted sub process len(results) : 15 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 15 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10582 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.0013256072998046875 nb_pixel_total : 19490 time to create 1 rle with old method : 0.026723384857177734 length of segment : 286 time for calcul the mask position with numpy : 0.04144620895385742 nb_pixel_total : 64205 time to create 1 rle with old method : 0.07133245468139648 length of segment : 508 time for calcul the mask position with numpy : 0.00630497932434082 nb_pixel_total : 404612 time to create 1 rle with new method : 0.03363823890686035 length of segment : 708 time for calcul the mask position with numpy : 0.006159543991088867 nb_pixel_total : 390525 time to create 1 rle with new method : 0.01627635955810547 length of segment : 664 time for calcul the mask position with numpy : 0.0007445812225341797 nb_pixel_total : 14658 time to create 1 rle with old method : 0.016988515853881836 length of segment : 292 time for calcul the mask position with numpy : 0.0014748573303222656 nb_pixel_total : 48079 time to create 1 rle with old method : 0.051217079162597656 length of segment : 632 time for calcul the mask position with numpy : 0.007947921752929688 nb_pixel_total : 421134 time to create 1 rle with new method : 0.1327953338623047 length of segment : 979 time for calcul the mask position with numpy : 0.010074615478515625 nb_pixel_total : 403934 time to create 1 rle with new method : 0.03160262107849121 length of segment : 644 time for calcul the mask position with numpy : 0.00033092498779296875 nb_pixel_total : 11566 time to create 1 rle with old method : 0.01245260238647461 length of segment : 147 time for calcul the mask position with numpy : 0.031258344650268555 nb_pixel_total : 1128260 time to create 1 rle with new method : 0.44040703773498535 length of segment : 2444 time for calcul the mask position with numpy : 0.000492095947265625 nb_pixel_total : 28674 time to create 1 rle with old method : 0.03082728385925293 length of segment : 217 time for calcul the mask position with numpy : 0.00045418739318847656 nb_pixel_total : 27779 time to create 1 rle with old method : 0.030834197998046875 length of segment : 138 time for calcul the mask position with numpy : 0.011510848999023438 nb_pixel_total : 478666 time to create 1 rle with new method : 0.03391838073730469 length of segment : 1083 time for calcul the mask position with numpy : 0.008157968521118164 nb_pixel_total : 427245 time to create 1 rle with new method : 0.022139310836791992 length of segment : 778 time for calcul the mask position with numpy : 0.00019979476928710938 nb_pixel_total : 7701 time to create 1 rle with old method : 0.00861978530883789 length of segment : 94 time for calcul the mask position with numpy : 0.0013339519500732422 nb_pixel_total : 73456 time to create 1 rle with old method : 0.07703042030334473 length of segment : 474 time for calcul the mask position with numpy : 0.0018222332000732422 nb_pixel_total : 111256 time to create 1 rle with old method : 0.11757779121398926 length of segment : 557 time for calcul the mask position with numpy : 0.00022912025451660156 nb_pixel_total : 8962 time to create 1 rle with old method : 0.0098876953125 length of segment : 146 time for calcul the mask position with numpy : 0.0002522468566894531 nb_pixel_total : 5977 time to create 1 rle with old method : 0.006612300872802734 length of segment : 124 time for calcul the mask position with numpy : 0.002145528793334961 nb_pixel_total : 127207 time to create 1 rle with old method : 0.13550567626953125 length of segment : 695 time for calcul the mask position with numpy : 0.0009424686431884766 nb_pixel_total : 56985 time to create 1 rle with old method : 0.060212135314941406 length of segment : 707 time for calcul the mask position with numpy : 0.005310535430908203 nb_pixel_total : 244441 time to create 1 rle with new method : 0.017247438430786133 length of segment : 1370 time for calcul the mask position with numpy : 0.0017807483673095703 nb_pixel_total : 123039 time to create 1 rle with old method : 0.13118815422058105 length of segment : 398 time for calcul the mask position with numpy : 0.001626729965209961 nb_pixel_total : 88882 time to create 1 rle with old method : 0.09444165229797363 length of segment : 441 time for calcul the mask position with numpy : 0.027099609375 nb_pixel_total : 890574 time to create 1 rle with new method : 0.2031419277191162 length of segment : 2842 time for calcul the mask position with numpy : 0.001459360122680664 nb_pixel_total : 91323 time to create 1 rle with old method : 0.09577608108520508 length of segment : 424 time for calcul the mask position with numpy : 0.002122640609741211 nb_pixel_total : 118981 time to create 1 rle with old method : 0.12467551231384277 length of segment : 588 time for calcul the mask position with numpy : 0.0006997585296630859 nb_pixel_total : 32037 time to create 1 rle with old method : 0.03358054161071777 length of segment : 334 time for calcul the mask position with numpy : 0.0016326904296875 nb_pixel_total : 89467 time to create 1 rle with old method : 0.0955352783203125 length of segment : 390 time for calcul the mask position with numpy : 0.0017404556274414062 nb_pixel_total : 75588 time to create 1 rle with old method : 0.09698319435119629 length of segment : 305 time for calcul the mask position with numpy : 0.024716854095458984 nb_pixel_total : 1091948 time to create 1 rle with new method : 0.3006269931793213 length of segment : 2129 time for calcul the mask position with numpy : 0.0071413516998291016 nb_pixel_total : 365541 time to create 1 rle with new method : 0.033240556716918945 length of segment : 638 time for calcul the mask position with numpy : 0.0013942718505859375 nb_pixel_total : 85324 time to create 1 rle with old method : 0.08980607986450195 length of segment : 324 time for calcul the mask position with numpy : 0.0004153251647949219 nb_pixel_total : 26356 time to create 1 rle with old method : 0.028408288955688477 length of segment : 195 time for calcul the mask position with numpy : 0.00043010711669921875 nb_pixel_total : 18890 time to create 1 rle with old method : 0.020575523376464844 length of segment : 144 time for calcul the mask position with numpy : 0.0018606185913085938 nb_pixel_total : 142300 time to create 1 rle with old method : 0.15048933029174805 length of segment : 294 time for calcul the mask position with numpy : 0.003773927688598633 nb_pixel_total : 169218 time to create 1 rle with new method : 0.006298065185546875 length of segment : 626 time for calcul the mask position with numpy : 0.0008916854858398438 nb_pixel_total : 37208 time to create 1 rle with old method : 0.039632320404052734 length of segment : 201 time for calcul the mask position with numpy : 0.0015153884887695312 nb_pixel_total : 66735 time to create 1 rle with old method : 0.07152605056762695 length of segment : 219 time for calcul the mask position with numpy : 0.0010540485382080078 nb_pixel_total : 41361 time to create 1 rle with old method : 0.04694819450378418 length of segment : 253 time for calcul the mask position with numpy : 0.029049158096313477 nb_pixel_total : 977664 time to create 1 rle with new method : 0.11817741394042969 length of segment : 2368 time for calcul the mask position with numpy : 0.00022029876708984375 nb_pixel_total : 2828 time to create 1 rle with old method : 0.003078460693359375 length of segment : 77 time for calcul the mask position with numpy : 0.0016903877258300781 nb_pixel_total : 76747 time to create 1 rle with old method : 0.08037662506103516 length of segment : 372 time for calcul the mask position with numpy : 0.0003104209899902344 nb_pixel_total : 20788 time to create 1 rle with old method : 0.022434473037719727 length of segment : 119 time for calcul the mask position with numpy : 0.00828695297241211 nb_pixel_total : 405116 time to create 1 rle with new method : 0.01359701156616211 length of segment : 600 time for calcul the mask position with numpy : 0.0005831718444824219 nb_pixel_total : 16875 time to create 1 rle with old method : 0.018083810806274414 length of segment : 135 time for calcul the mask position with numpy : 0.0023474693298339844 nb_pixel_total : 126387 time to create 1 rle with old method : 0.13397002220153809 length of segment : 474 time for calcul the mask position with numpy : 0.008220911026000977 nb_pixel_total : 337527 time to create 1 rle with new method : 0.014946937561035156 length of segment : 1509 time for calcul the mask position with numpy : 0.0005812644958496094 nb_pixel_total : 24400 time to create 1 rle with old method : 0.025763273239135742 length of segment : 157 time for calcul the mask position with numpy : 0.004231929779052734 nb_pixel_total : 191116 time to create 1 rle with new method : 0.006949186325073242 length of segment : 296 time for calcul the mask position with numpy : 0.034895896911621094 nb_pixel_total : 662364 time to create 1 rle with new method : 0.6251463890075684 length of segment : 2236 time for calcul the mask position with numpy : 0.0031499862670898438 nb_pixel_total : 121337 time to create 1 rle with old method : 0.13167190551757812 length of segment : 556 time for calcul the mask position with numpy : 0.0002980232238769531 nb_pixel_total : 4849 time to create 1 rle with old method : 0.005830049514770508 length of segment : 122 time for calcul the mask position with numpy : 0.0003075599670410156 nb_pixel_total : 4502 time to create 1 rle with old method : 0.005075931549072266 length of segment : 96 time for calcul the mask position with numpy : 0.00048613548278808594 nb_pixel_total : 20163 time to create 1 rle with old method : 0.022742033004760742 length of segment : 64 time for calcul the mask position with numpy : 0.0002925395965576172 nb_pixel_total : 14536 time to create 1 rle with old method : 0.016321420669555664 length of segment : 193 time for calcul the mask position with numpy : 0.002333402633666992 nb_pixel_total : 96022 time to create 1 rle with old method : 0.10195755958557129 length of segment : 336 time for calcul the mask position with numpy : 0.0005526542663574219 nb_pixel_total : 15924 time to create 1 rle with old method : 0.01710367202758789 length of segment : 145 time for calcul the mask position with numpy : 0.0017771720886230469 nb_pixel_total : 50439 time to create 1 rle with old method : 0.054221391677856445 length of segment : 329 time for calcul the mask position with numpy : 0.004674434661865234 nb_pixel_total : 152217 time to create 1 rle with new method : 0.010573863983154297 length of segment : 523 time for calcul the mask position with numpy : 0.0004253387451171875 nb_pixel_total : 12140 time to create 1 rle with old method : 0.013451814651489258 length of segment : 109 time for calcul the mask position with numpy : 0.009031295776367188 nb_pixel_total : 411104 time to create 1 rle with new method : 0.014759302139282227 length of segment : 797 time for calcul the mask position with numpy : 0.0014717578887939453 nb_pixel_total : 61019 time to create 1 rle with old method : 0.06512641906738281 length of segment : 360 time for calcul the mask position with numpy : 0.0004134178161621094 nb_pixel_total : 28186 time to create 1 rle with old method : 0.02940845489501953 length of segment : 230 time for calcul the mask position with numpy : 0.000858306884765625 nb_pixel_total : 20623 time to create 1 rle with old method : 0.022182464599609375 length of segment : 245 time for calcul the mask position with numpy : 0.0010094642639160156 nb_pixel_total : 19489 time to create 1 rle with old method : 0.02080535888671875 length of segment : 286 time for calcul the mask position with numpy : 0.07532715797424316 nb_pixel_total : 64276 time to create 1 rle with old method : 0.06956243515014648 length of segment : 508 time for calcul the mask position with numpy : 0.007028341293334961 nb_pixel_total : 404592 time to create 1 rle with new method : 0.015106439590454102 length of segment : 708 time for calcul the mask position with numpy : 0.005568265914916992 nb_pixel_total : 390508 time to create 1 rle with new method : 0.013484954833984375 length of segment : 664 time for calcul the mask position with numpy : 0.0006604194641113281 nb_pixel_total : 14682 time to create 1 rle with old method : 0.017282962799072266 length of segment : 292 time for calcul the mask position with numpy : 0.001775503158569336 nb_pixel_total : 47977 time to create 1 rle with old method : 0.053037405014038086 length of segment : 631 time for calcul the mask position with numpy : 0.006765127182006836 nb_pixel_total : 421106 time to create 1 rle with new method : 0.01687026023864746 length of segment : 978 time for calcul the mask position with numpy : 0.0066699981689453125 nb_pixel_total : 403987 time to create 1 rle with new method : 0.017802715301513672 length of segment : 644 time for calcul the mask position with numpy : 0.0002772808074951172 nb_pixel_total : 11567 time to create 1 rle with old method : 0.013179540634155273 length of segment : 147 time for calcul the mask position with numpy : 0.034119367599487305 nb_pixel_total : 1128120 time to create 1 rle with new method : 0.448040246963501 length of segment : 2444 time for calcul the mask position with numpy : 0.0004603862762451172 nb_pixel_total : 28683 time to create 1 rle with old method : 0.03252720832824707 length of segment : 217 time for calcul the mask position with numpy : 0.0007147789001464844 nb_pixel_total : 27769 time to create 1 rle with old method : 0.03117966651916504 length of segment : 138 time for calcul the mask position with numpy : 0.010808467864990234 nb_pixel_total : 478653 time to create 1 rle with new method : 0.029775381088256836 length of segment : 1083 time for calcul the mask position with numpy : 0.0068743228912353516 nb_pixel_total : 427272 time to create 1 rle with new method : 0.01708245277404785 length of segment : 778 time for calcul the mask position with numpy : 0.0003445148468017578 nb_pixel_total : 7701 time to create 1 rle with old method : 0.00863337516784668 length of segment : 94 time for calcul the mask position with numpy : 0.002766847610473633 nb_pixel_total : 111220 time to create 1 rle with old method : 0.12215518951416016 length of segment : 557 time for calcul the mask position with numpy : 0.031888723373413086 nb_pixel_total : 1041869 time to create 1 rle with new method : 0.23985815048217773 length of segment : 2692 time for calcul the mask position with numpy : 0.0002079010009765625 nb_pixel_total : 8977 time to create 1 rle with old method : 0.010301351547241211 length of segment : 146 time for calcul the mask position with numpy : 0.0004744529724121094 nb_pixel_total : 5974 time to create 1 rle with old method : 0.006903171539306641 length of segment : 124 time for calcul the mask position with numpy : 0.0008215904235839844 nb_pixel_total : 56962 time to create 1 rle with old method : 0.06407022476196289 length of segment : 707 time for calcul the mask position with numpy : 0.002577543258666992 nb_pixel_total : 88737 time to create 1 rle with old method : 0.09727025032043457 length of segment : 451 time for calcul the mask position with numpy : 0.001920938491821289 nb_pixel_total : 128919 time to create 1 rle with old method : 0.15066790580749512 length of segment : 703 time for calcul the mask position with numpy : 0.006547212600708008 nb_pixel_total : 244444 time to create 1 rle with new method : 0.013753652572631836 length of segment : 1371 time for calcul the mask position with numpy : 0.0015151500701904297 nb_pixel_total : 122993 time to create 1 rle with old method : 0.13720989227294922 length of segment : 397 time for calcul the mask position with numpy : 0.0022792816162109375 nb_pixel_total : 91296 time to create 1 rle with old method : 0.10100293159484863 length of segment : 424 time for calcul the mask position with numpy : 0.0025391578674316406 nb_pixel_total : 118976 time to create 1 rle with old method : 0.12886452674865723 length of segment : 587 time for calcul the mask position with numpy : 0.0012154579162597656 nb_pixel_total : 87141 time to create 1 rle with old method : 0.09520602226257324 length of segment : 416 time for calcul the mask position with numpy : 0.0006136894226074219 nb_pixel_total : 32040 time to create 1 rle with old method : 0.03548264503479004 length of segment : 334 time for calcul the mask position with numpy : 0.0021021366119384766 nb_pixel_total : 75578 time to create 1 rle with old method : 0.08475065231323242 length of segment : 305 time for calcul the mask position with numpy : 0.026012420654296875 nb_pixel_total : 1091917 time to create 1 rle with new method : 0.11209297180175781 length of segment : 2129 time for calcul the mask position with numpy : 0.010216474533081055 nb_pixel_total : 365556 time to create 1 rle with new method : 0.07098698616027832 length of segment : 638 time for calcul the mask position with numpy : 0.0015251636505126953 nb_pixel_total : 85327 time to create 1 rle with old method : 0.09400725364685059 length of segment : 324 time for calcul the mask position with numpy : 0.0006122589111328125 nb_pixel_total : 26362 time to create 1 rle with old method : 0.03007364273071289 length of segment : 195 time for calcul the mask position with numpy : 0.0004229545593261719 nb_pixel_total : 18892 time to create 1 rle with old method : 0.02185511589050293 length of segment : 144 time for calcul the mask position with numpy : 0.002457141876220703 nb_pixel_total : 142221 time to create 1 rle with old method : 0.15442872047424316 length of segment : 294 time for calcul the mask position with numpy : 0.003219127655029297 nb_pixel_total : 169248 time to create 1 rle with new method : 0.005225181579589844 length of segment : 626 time for calcul the mask position with numpy : 0.0008256435394287109 nb_pixel_total : 37207 time to create 1 rle with old method : 0.04133415222167969 length of segment : 201 time for calcul the mask position with numpy : 0.001421213150024414 nb_pixel_total : 66751 time to create 1 rle with old method : 0.07420730590820312 length of segment : 219 time for calcul the mask position with numpy : 0.0010149478912353516 nb_pixel_total : 41538 time to create 1 rle with old method : 0.04694485664367676 length of segment : 251 time for calcul the mask position with numpy : 0.023997068405151367 nb_pixel_total : 974163 time to create 1 rle with new method : 0.2834019660949707 length of segment : 2374 time for calcul the mask position with numpy : 0.00018739700317382812 nb_pixel_total : 2813 time to create 1 rle with old method : 0.0032618045806884766 length of segment : 77 time for calcul the mask position with numpy : 0.0015649795532226562 nb_pixel_total : 77377 time to create 1 rle with old method : 0.0844111442565918 length of segment : 379 time for calcul the mask position with numpy : 0.0002779960632324219 nb_pixel_total : 20783 time to create 1 rle with old method : 0.023842811584472656 length of segment : 119 time for calcul the mask position with numpy : 0.007533073425292969 nb_pixel_total : 405121 time to create 1 rle with new method : 0.010668277740478516 length of segment : 600 time for calcul the mask position with numpy : 0.0005507469177246094 nb_pixel_total : 16875 time to create 1 rle with old method : 0.01893305778503418 length of segment : 135 time for calcul the mask position with numpy : 0.002010345458984375 nb_pixel_total : 126390 time to create 1 rle with old method : 0.13733410835266113 length of segment : 474 time for calcul the mask position with numpy : 0.0074596405029296875 nb_pixel_total : 337595 time to create 1 rle with new method : 0.012354135513305664 length of segment : 1509 time for calcul the mask position with numpy : 0.0005440711975097656 nb_pixel_total : 24403 time to create 1 rle with old method : 0.027394533157348633 length of segment : 157 time for calcul the mask position with numpy : 0.0037679672241210938 nb_pixel_total : 191118 time to create 1 rle with new method : 0.00545048713684082 length of segment : 296 time for calcul the mask position with numpy : 0.030603408813476562 nb_pixel_total : 662811 time to create 1 rle with new method : 0.29291462898254395 length of segment : 2237 time for calcul the mask position with numpy : 0.0026428699493408203 nb_pixel_total : 121340 time to create 1 rle with old method : 0.13020539283752441 length of segment : 556 time for calcul the mask position with numpy : 0.00027871131896972656 nb_pixel_total : 4839 time to create 1 rle with old method : 0.005961894989013672 length of segment : 122 time for calcul the mask position with numpy : 0.00028061866760253906 nb_pixel_total : 4501 time to create 1 rle with old method : 0.005247592926025391 length of segment : 96 time for calcul the mask position with numpy : 0.0004534721374511719 nb_pixel_total : 20163 time to create 1 rle with old method : 0.02249312400817871 length of segment : 64 time for calcul the mask position with numpy : 0.00027942657470703125 nb_pixel_total : 14533 time to create 1 rle with old method : 0.01677680015563965 length of segment : 193 time for calcul the mask position with numpy : 0.0022590160369873047 nb_pixel_total : 96023 time to create 1 rle with old method : 0.10388588905334473 length of segment : 336 time for calcul the mask position with numpy : 0.0004901885986328125 nb_pixel_total : 15925 time to create 1 rle with old method : 0.017895221710205078 length of segment : 145 time for calcul the mask position with numpy : 0.0016808509826660156 nb_pixel_total : 50438 time to create 1 rle with old method : 0.05580949783325195 length of segment : 329 time for calcul the mask position with numpy : 0.004229307174682617 nb_pixel_total : 152221 time to create 1 rle with new method : 0.008705854415893555 length of segment : 523 time for calcul the mask position with numpy : 0.0003495216369628906 nb_pixel_total : 12142 time to create 1 rle with old method : 0.014087200164794922 length of segment : 109 time for calcul the mask position with numpy : 0.00776362419128418 nb_pixel_total : 411117 time to create 1 rle with new method : 0.01195383071899414 length of segment : 797 time for calcul the mask position with numpy : 0.0013844966888427734 nb_pixel_total : 61019 time to create 1 rle with old method : 0.06748723983764648 length of segment : 360 time for calcul the mask position with numpy : 0.0004019737243652344 nb_pixel_total : 28187 time to create 1 rle with old method : 0.031004905700683594 length of segment : 230 time for calcul the mask position with numpy : 0.0007555484771728516 nb_pixel_total : 20622 time to create 1 rle with old method : 0.023001909255981445 length of segment : 245 time for calcul the mask position with numpy : 0.045591115951538086 nb_pixel_total : 1244009 time to create 1 rle with new method : 0.6412308216094971 length of segment : 3125 time for calcul the mask position with numpy : 0.004537105560302734 nb_pixel_total : 318702 time to create 1 rle with new method : 0.010906457901000977 length of segment : 790 time for calcul the mask position with numpy : 0.00032591819763183594 nb_pixel_total : 18520 time to create 1 rle with old method : 0.021376609802246094 length of segment : 181 time for calcul the mask position with numpy : 0.019954681396484375 nb_pixel_total : 1175494 time to create 1 rle with new method : 0.2557535171508789 length of segment : 2161 time for calcul the mask position with numpy : 0.0007169246673583984 nb_pixel_total : 47946 time to create 1 rle with old method : 0.051552534103393555 length of segment : 351 time for calcul the mask position with numpy : 0.020705223083496094 nb_pixel_total : 1226659 time to create 1 rle with new method : 0.1782522201538086 length of segment : 1177 time for calcul the mask position with numpy : 0.0786428451538086 nb_pixel_total : 1940755 time to create 1 rle with new method : 0.7277874946594238 length of segment : 2925 time for calcul the mask position with numpy : 0.0013012886047363281 nb_pixel_total : 84119 time to create 1 rle with old method : 0.0958092212677002 length of segment : 416 time for calcul the mask position with numpy : 0.0026650428771972656 nb_pixel_total : 183260 time to create 1 rle with new method : 0.006920814514160156 length of segment : 521 time for calcul the mask position with numpy : 0.0017948150634765625 nb_pixel_total : 128608 time to create 1 rle with old method : 0.142411470413208 length of segment : 485 time for calcul the mask position with numpy : 0.019443511962890625 nb_pixel_total : 1037172 time to create 1 rle with new method : 0.10864663124084473 length of segment : 2322 time for calcul the mask position with numpy : 0.0011327266693115234 nb_pixel_total : 86048 time to create 1 rle with old method : 0.0940549373626709 length of segment : 414 time for calcul the mask position with numpy : 0.004876136779785156 nb_pixel_total : 348897 time to create 1 rle with new method : 0.01303863525390625 length of segment : 1164 time for calcul the mask position with numpy : 0.0009362697601318359 nb_pixel_total : 80558 time to create 1 rle with old method : 0.09126663208007812 length of segment : 241 time for calcul the mask position with numpy : 0.00040650367736816406 nb_pixel_total : 9204 time to create 1 rle with old method : 0.010540008544921875 length of segment : 135 time for calcul the mask position with numpy : 0.22173428535461426 nb_pixel_total : 2701124 time to create 1 rle with new method : 0.5429308414459229 length of segment : 3029 time for calcul the mask position with numpy : 0.0013806819915771484 nb_pixel_total : 68921 time to create 1 rle with old method : 0.0776374340057373 length of segment : 186 time for calcul the mask position with numpy : 0.0002484321594238281 nb_pixel_total : 5974 time to create 1 rle with old method : 0.0070188045501708984 length of segment : 103 time for calcul the mask position with numpy : 0.010445356369018555 nb_pixel_total : 650870 time to create 1 rle with new method : 0.07235026359558105 length of segment : 1734 time for calcul the mask position with numpy : 0.004586935043334961 nb_pixel_total : 246481 time to create 1 rle with new method : 0.009701013565063477 length of segment : 534 time for calcul the mask position with numpy : 0.0010569095611572266 nb_pixel_total : 63687 time to create 1 rle with old method : 0.07246017456054688 length of segment : 243 time spent for convertir_results : 32.998175621032715 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 150 chid ids of type : 3663 Number RLEs to save : 95426 save missing photos in datou_result : time spend for datou_step_exec : 106.10865473747253 time spend to save output : 5.170633316040039 total time spend for step 1 : 111.27928805351257 step2:brightness Fri Sep 26 10:31:19 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1758875363_4185575_1386052877_1ec34129bf0d54ff7fdcfb15578a4ab7.jpg treat image : temp/1758875363_4185575_1386052814_e15a80fcd0a96c572ecc7e562febea96.jpg treat image : temp/1758875363_4185575_1386052750_91a1a2e8339b87a3f18054214b570365.jpg treat image : temp/1758875363_4185575_1386052651_16fab6254bfbf4d3edeb6432f962bb84.jpg treat image : temp/1758875363_4185575_1386052547_edf5b29219c6d609c6865c87dfdec492.jpg treat image : temp/1758875363_4185575_1386052479_1ed2ec29d9d843c403d14ce8efadc8f8.jpg treat image : temp/1758875363_4185575_1386049645_996ba35e66df1d993267147b532d6a76.jpg treat image : temp/1758875363_4185575_1386049638_72aafc3860d22b74ed6f3a670941e651.jpg treat image : temp/1758875363_4185575_1386049619_ee6b0ab5fe6c9e96d9f464e9b1aea831.jpg treat image : temp/1758875363_4185575_1386049588_efe94e0f6eed9eebca9ccfcb33de407b.jpg treat image : temp/1758875363_4185575_1386049569_dd6a9a4752de3fd880b7ca8b4a28259e.jpg treat image : temp/1758875363_4185575_1386049568_12ad5e14152cb315dc90a2094d5cb7b1.jpg treat image : temp/1758875363_4185575_1386048608_db5253df789e5e9655bc05cb179451a5.jpg treat image : temp/1758875363_4185575_1386048577_a4455050ef07ca078b71ff28e85d9cd1.jpg treat image : temp/1758875363_4185575_1386048569_0a4fd19d2357a5438009bd6c1fe1dd84.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 15 time used for this insertion : 0.01290583610534668 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 15 time used for this insertion : 0.012800216674804688 save missing photos in datou_result : time spend for datou_step_exec : 22.830098152160645 time spend to save output : 0.030489444732666016 total time spend for step 2 : 22.86058759689331 step3:blur_detection Fri Sep 26 10:31:41 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1758875363_4185575_1386052877_1ec34129bf0d54ff7fdcfb15578a4ab7.jpg resize: (3040, 4056) 1386052877 0.042370264104989584 treat image : temp/1758875363_4185575_1386052814_e15a80fcd0a96c572ecc7e562febea96.jpg resize: (3040, 4056) 1386052814 0.004459216212001633 treat image : temp/1758875363_4185575_1386052750_91a1a2e8339b87a3f18054214b570365.jpg resize: (3040, 4056) 1386052750 0.17099418973702882 treat image : temp/1758875363_4185575_1386052651_16fab6254bfbf4d3edeb6432f962bb84.jpg resize: (3040, 4056) 1386052651 0.45489254576316085 treat image : temp/1758875363_4185575_1386052547_edf5b29219c6d609c6865c87dfdec492.jpg resize: (3040, 4056) 1386052547 -1.6497149410629899 treat image : temp/1758875363_4185575_1386052479_1ed2ec29d9d843c403d14ce8efadc8f8.jpg resize: (3040, 4056) 1386052479 1.0257004945970205 treat image : temp/1758875363_4185575_1386049645_996ba35e66df1d993267147b532d6a76.jpg resize: (3040, 4056) 1386049645 0.042370264104989584 treat image : temp/1758875363_4185575_1386049638_72aafc3860d22b74ed6f3a670941e651.jpg resize: (3040, 4056) 1386049638 0.004459216212001633 treat image : temp/1758875363_4185575_1386049619_ee6b0ab5fe6c9e96d9f464e9b1aea831.jpg resize: (3040, 4056) 1386049619 0.17099418973702882 treat image : temp/1758875363_4185575_1386049588_efe94e0f6eed9eebca9ccfcb33de407b.jpg resize: (3040, 4056) 1386049588 0.45489254576316085 treat image : temp/1758875363_4185575_1386049569_dd6a9a4752de3fd880b7ca8b4a28259e.jpg resize: (3040, 4056) 1386049569 -1.6497149410629899 treat image : temp/1758875363_4185575_1386049568_12ad5e14152cb315dc90a2094d5cb7b1.jpg resize: (3040, 4056) 1386049568 1.0257004945970205 treat image : temp/1758875363_4185575_1386048608_db5253df789e5e9655bc05cb179451a5.jpg resize: (3040, 4056) 1386048608 0.43780932820796015 treat image : temp/1758875363_4185575_1386048577_a4455050ef07ca078b71ff28e85d9cd1.jpg resize: (3040, 4056) 1386048577 0.6391297616167821 treat image : temp/1758875363_4185575_1386048569_0a4fd19d2357a5438009bd6c1fe1dd84.jpg resize: (3040, 4056) 1386048569 -0.37773187146262144 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 15 time used for this insertion : 0.012958526611328125 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 15 time used for this insertion : 0.012445449829101562 save missing photos in datou_result : time spend for datou_step_exec : 81.29380655288696 time spend to save output : 0.029796838760375977 total time spend for step 3 : 81.32360339164734 step4:crop_condition Fri Sep 26 10:33:03 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 15 ! batch 1 Loaded 150 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 16 About to insert : list_path_to_insert length 16 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 28 About to insert : list_path_to_insert length 28 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 33 About to insert : list_path_to_insert length 33 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1386052877, 1386052814, 1386052750, 1386052651, 1386052547, 1386052479, 1386049645, 1386049638, 1386049619, 1386049588, 1386049569, 1386049568, 1386048608, 1386048577, 1386048569] Looping around the photos to save general results len do output : 90 /-3976579526Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579530Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579541Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579543Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579561Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579568Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579582Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579591Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579595Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579606Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579610Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579625Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579632Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579646Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579658Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579664Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579559Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579573Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579575Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579585Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579623Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579637Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579639Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579649Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579654Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579535Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579536Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579545Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579546Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579550Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579552Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579562Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579567Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579570Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579571Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579580Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579583Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579587Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579600Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579601Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579609Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579611Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579612Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579614Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579626Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579631Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579634Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579635Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579644Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579647Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579651Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579661Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579672Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579527Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579528Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579529Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579531Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579532Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579540Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579542Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579544Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579553Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579555Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579556Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579557Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579560Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579581Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579586Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579592Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579593Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579594Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579596Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579597Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579604Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579607Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579608Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579617Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579619Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579620Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579621Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579624Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579645Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579650Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579660Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579667Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579669Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579572Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579636Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579563Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976579627Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052877', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052814', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052750', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052651', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052547', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052479', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049645', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049638', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049619', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049588', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049569', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049568', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048608', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048577', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048569', None, None, None, None, None, '3784106') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 285 time used for this insertion : 0.03930377960205078 save_final save missing photos in datou_result : time spend for datou_step_exec : 95.07864904403687 time spend to save output : 0.07459902763366699 total time spend for step 4 : 95.15324807167053 step5:thcl Fri Sep 26 10:34:38 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 1 we are using the classfication for multi_thcl [2456] time to import caffe and check if the image exist : 0.01695990562438965 time to convert the images to numpy array : 0.14134955406188965 time to import caffe and check if the image exist : 0.016937732696533203 time to convert the images to numpy array : 0.38300490379333496 time to import caffe and check if the image exist : 0.01703047752380371 time to convert the images to numpy array : 0.4901008605957031 time to import caffe and check if the image exist : 0.019763946533203125 time to convert the images to numpy array : 0.604494571685791 time to import caffe and check if the image exist : 0.019356489181518555 time to convert the images to numpy array : 0.674706220626831 time to import caffe and check if the image exist : 0.0203554630279541 time to convert the images to numpy array : 0.7937779426574707 time to import caffe and check if the image exist : 0.018203258514404297 time to convert the images to numpy array : 0.8558349609375 time to import caffe and check if the image exist : 0.022304296493530273 time to convert the images to numpy array : 0.9671223163604736 time to import caffe and check if the image exist : 0.01198267936706543 time to convert the images to numpy array : 1.11686110496521 time to import caffe and check if the image exist : 0.0169064998626709 time to convert the images to numpy array : 1.1810224056243896 total time to convert the images to numpy array : 1.6534316539764404 list photo_ids error: [] list photo_ids correct : [-3976579645, -3976579650, -3976579660, -3976579667, -3976579669, -3976579572, -3976579636, -3976579563, -3976579627, -3976579597, -3976579604, -3976579607, -3976579608, -3976579617, -3976579619, -3976579620, -3976579621, -3976579624, -3976579545, -3976579546, -3976579550, -3976579552, -3976579562, -3976579567, -3976579570, -3976579571, -3976579580, -3976579556, -3976579557, -3976579560, -3976579581, -3976579586, -3976579592, -3976579593, -3976579594, -3976579596, -3976579528, -3976579529, -3976579531, -3976579532, -3976579540, -3976579542, -3976579544, -3976579553, -3976579555, -3976579583, -3976579587, -3976579600, -3976579601, -3976579609, -3976579611, -3976579612, -3976579614, -3976579626, -3976579631, -3976579634, -3976579635, -3976579644, -3976579647, -3976579651, -3976579661, -3976579672, -3976579527, -3976579606, -3976579610, -3976579625, -3976579632, -3976579646, -3976579658, -3976579664, -3976579559, -3976579573, -3976579526, -3976579530, -3976579541, -3976579543, -3976579561, -3976579568, -3976579582, -3976579591, -3976579595, -3976579575, -3976579585, -3976579623, -3976579637, -3976579639, -3976579649, -3976579654, -3976579535, -3976579536] number of photos to traite : 90 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 6800 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/:/home/admin/workarea/git/apy/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 6581 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 2.0480427742004395 time used to do the prediction : 0.23577356338500977 we don't save the descriptors for this thcl 2456 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 9.5367431640625e-06 save missing photos in datou_result : time spend for datou_step_exec : 9.003628969192505 time spend to save output : 0.00027251243591308594 total time spend for step 5 : 9.003901481628418 step6:tfhub_classification2 Fri Sep 26 10:34:47 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou_step TFHub with tf2 ! we are using the classfication for only one thcl 3501 begin to check gpu status inside check gpu memory 2025-09-26 10:34:51.537978: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-26 10:34:51.539793: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 10:34:51.539975: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 10:34:51.540223: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 10:34:51.558896: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 10:34:51.559002: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 10:34:51.589456: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 10:34:51.593953: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 10:34:51.655700: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 10:34:51.657194: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 10:34:51.658301: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-26 10:34:51.688743: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-26 10:34:51.690386: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f5c8c000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-26 10:34:51.690432: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-26 10:34:51.695345: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x10fe7c70 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-26 10:34:51.695368: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-26 10:34:51.696715: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 10:34:51.696801: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 10:34:51.696819: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 10:34:51.696924: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 10:34:51.696944: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 10:34:51.696969: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 10:34:51.696994: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 10:34:51.697040: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 10:34:51.698087: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 10:34:51.698497: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 10:34:51.698665: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-26 10:34:51.698677: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-26 10:34:51.698688: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-26 10:34:51.699716: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3096 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) l 3637 free memory gpu now : 6579 max_wait_temp : 1 max_wait : 5 1 Physical GPUs, 1 Logical GPUs tagging for thcl : 3501 To do loadFromThcl(), then load ParamDescType : thcl3501 thcls : [{'id': 3501, 'mtr_user_id': 31, 'name': 'learn_papier_mixte_centre_tf_b4_220223_3', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '9087724,9087727,9087725,9087729,9087726,9087723,9087722,9087720,9087721,9087728', 'photo_hashtag_type': 4539, 'photo_desc_type': 5759, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 3501, 'mtr_user_id': 31, 'name': 'learn_papier_mixte_centre_tf_b4_220223_3', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '9087724,9087727,9087725,9087729,9087726,9087723,9087722,9087720,9087721,9087728', 'photo_hashtag_type': 4539, 'photo_desc_type': 5759, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5759 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5759, 'learn_papier_mixte_centre_tf_b4_220223_3', 1792, 1792, 'learn_papier_mixte_centre_tf_b4_220223_3', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 2, 22, 13, 54, 32), datetime.datetime(2023, 2, 22, 13, 54, 32)) model_name : learn_papier_mixte_centre_tf_b4_220223_3 model_param file didn't exist model_name : learn_papier_mixte_centre_tf_b4_220223_3 model_type : tf_classification2 list file need : ['Confusion_Matrix.png', 'Precision_Recall_Carton_brun.jpg', 'Precision_Recall_Carton_gris.jpg', 'Precision_Recall_Teint_Dans_La_Masse.jpg', 'Precision_Recall_autre_refus.jpg', 'Precision_Recall_cartonnette.jpg', 'Precision_Recall_environnement.jpg', 'Precision_Recall_kraft.jpg', 'Precision_Recall_metal.jpg', 'Precision_Recall_papier.jpg', 'Precision_Recall_plastique.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file exist in s3 : ['Confusion_Matrix.png', 'Precision_Recall_Carton_brun.jpg', 'Precision_Recall_Carton_gris.jpg', 'Precision_Recall_Teint_Dans_La_Masse.jpg', 'Precision_Recall_autre_refus.jpg', 'Precision_Recall_cartonnette.jpg', 'Precision_Recall_environnement.jpg', 'Precision_Recall_kraft.jpg', 'Precision_Recall_metal.jpg', 'Precision_Recall_papier.jpg', 'Precision_Recall_plastique.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file manque in s3 : [] local folder : /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3 /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Confusion_Matrix.png size_local : 121269 size in s3 : 121269 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:17 Confusion_Matrix.png already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_Carton_brun.jpg size_local : 76104 size in s3 : 76104 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_Carton_brun.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_Carton_gris.jpg size_local : 82912 size in s3 : 82912 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_Carton_gris.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_Teint_Dans_La_Masse.jpg size_local : 75423 size in s3 : 75423 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:25 Precision_Recall_Teint_Dans_La_Masse.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_autre_refus.jpg size_local : 78883 size in s3 : 78883 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_autre_refus.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_cartonnette.jpg size_local : 81791 size in s3 : 81791 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:30 Precision_Recall_cartonnette.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_environnement.jpg size_local : 59936 size in s3 : 59936 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:17 Precision_Recall_environnement.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_kraft.jpg size_local : 80280 size in s3 : 80280 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:25 Precision_Recall_kraft.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_metal.jpg size_local : 77736 size in s3 : 77736 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_metal.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_papier.jpg size_local : 69041 size in s3 : 69041 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_papier.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_plastique.jpg size_local : 70770 size in s3 : 70770 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:17 Precision_Recall_plastique.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Result_Summary.txt size_local : 1092 size in s3 : 1092 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Result_Summary.txt already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/checkpoint size_local : 99 size in s3 : 99 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 checkpoint already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_checkpoint.ckpt.data-00000-of-00002 size_local : 429801 size in s3 : 429801 create time local : 2023-03-03 12:26:16 create time in s3 : 2023-02-22 12:54:30 model_checkpoint.ckpt.data-00000-of-00002 already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_checkpoint.ckpt.data-00001-of-00002 size_local : 141033188 size in s3 : 141033188 create time local : 2023-03-03 12:26:17 create time in s3 : 2023-02-22 12:54:25 model_checkpoint.ckpt.data-00001-of-00002 already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_checkpoint.ckpt.index size_local : 85618 size in s3 : 85618 create time local : 2023-03-03 12:26:17 create time in s3 : 2023-02-22 12:54:30 model_checkpoint.ckpt.index already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_weights.h5 size_local : 71289448 size in s3 : 71289448 create time local : 2023-03-03 12:26:18 create time in s3 : 2023-02-22 12:54:17 model_weights.h5 already exist and didn't need to update desc size : 1792 Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= module (KerasLayer) (None, 1792) 17673816 _________________________________________________________________ learn_papier_mixte_centre_tf (None, 10) 17930 ================================================================= Total params: 17,691,746 Trainable params: 17,930 Non-trainable params: 17,673,816 _________________________________________________________________ Loading Weights... time used to create the model : 18.54168701171875 time used to load_weights : 0.3162541389465332 0it [00:00, ?it/s]temp/1758875363_4185575_1386052877_1ec34129bf0d54ff7fdcfb15578a4ab7_rle_crop_3976579526_0.png temp/1758875363_4185575_1386052877_1ec34129bf0d54ff7fdcfb15578a4ab7_rle_crop_3976579530_0.png temp/1758875363_4185575_1386052814_e15a80fcd0a96c572ecc7e562febea96_rle_crop_3976579541_0.png temp/1758875363_4185575_1386052814_e15a80fcd0a96c572ecc7e562febea96_rle_crop_3976579543_0.png temp/1758875363_4185575_1386052651_16fab6254bfbf4d3edeb6432f962bb84_rle_crop_3976579561_0.png temp/1758875363_4185575_1386052547_edf5b29219c6d609c6865c87dfdec492_rle_crop_3976579568_0.png temp/1758875363_4185575_1386052479_1ed2ec29d9d843c403d14ce8efadc8f8_rle_crop_3976579582_0.png temp/1758875363_4185575_1386049645_996ba35e66df1d993267147b532d6a76_rle_crop_3976579591_0.png temp/1758875363_4185575_1386049645_996ba35e66df1d993267147b532d6a76_rle_crop_3976579595_0.png temp/1758875363_4185575_1386049638_72aafc3860d22b74ed6f3a670941e651_rle_crop_3976579606_0.png temp/1758875363_4185575_1386049638_72aafc3860d22b74ed6f3a670941e651_rle_crop_3976579610_0.png temp/1758875363_4185575_1386049588_efe94e0f6eed9eebca9ccfcb33de407b_rle_crop_3976579625_0.png temp/1758875363_4185575_1386049569_dd6a9a4752de3fd880b7ca8b4a28259e_rle_crop_3976579632_0.png temp/1758875363_4185575_1386049568_12ad5e14152cb315dc90a2094d5cb7b1_rle_crop_3976579646_0.png temp/1758875363_4185575_1386048577_a4455050ef07ca078b71ff28e85d9cd1_rle_crop_3976579658_0.png temp/1758875363_4185575_1386048577_a4455050ef07ca078b71ff28e85d9cd1_rle_crop_3976579664_0.png temp/1758875363_4185575_1386052651_16fab6254bfbf4d3edeb6432f962bb84_rle_crop_3976579559_0.png temp/1758875363_4185575_1386052547_edf5b29219c6d609c6865c87dfdec492_rle_crop_3976579573_0.png temp/1758875363_4185575_1386052547_edf5b29219c6d609c6865c87dfdec492_rle_crop_3976579575_0.png temp/1758875363_4185575_1386052479_1ed2ec29d9d843c403d14ce8efadc8f8_rle_crop_3976579585_0.png temp/1758875363_4185575_1386049588_efe94e0f6eed9eebca9ccfcb33de407b_rle_crop_3976579623_0.png temp/1758875363_4185575_1386049569_dd6a9a4752de3fd880b7ca8b4a28259e_rle_crop_3976579637_0.png temp/1758875363_4185575_1386049569_dd6a9a4752de3fd880b7ca8b4a28259e_rle_crop_3976579639_0.png temp/1758875363_4185575_1386049568_12ad5e14152cb315dc90a2094d5cb7b1_rle_crop_3976579649_0.png temp/1758875363_4185575_1386048608_db5253df789e5e9655bc05cb179451a5_rle_crop_3976579654_0.png temp/1758875363_4185575_1386052877_1ec34129bf0d54ff7fdcfb15578a4ab7_rle_crop_3976579535_0.png temp/1758875363_4185575_1386052877_1ec34129bf0d54ff7fdcfb15578a4ab7_rle_crop_3976579536_0.png temp/1758875363_4185575_1386052814_e15a80fcd0a96c572ecc7e562febea96_rle_crop_3976579545_0.png temp/1758875363_4185575_1386052814_e15a80fcd0a96c572ecc7e562febea96_rle_crop_3976579546_0.png temp/1758875363_4185575_1386052814_e15a80fcd0a96c572ecc7e562febea96_rle_crop_3976579550_0.png temp/1758875363_4185575_1386052814_e15a80fcd0a96c572ecc7e562febea96_rle_crop_3976579552_0.png temp/1758875363_4185575_1386052651_16fab6254bfbf4d3edeb6432f962bb84_rle_crop_3976579562_0.png temp/1758875363_4185575_1386052651_16fab6254bfbf4d3edeb6432f962bb84_rle_crop_3976579567_0.png temp/1758875363_4185575_1386052547_edf5b29219c6d609c6865c87dfdec492_rle_crop_3976579570_0.png temp/1758875363_4185575_1386052547_edf5b29219c6d609c6865c87dfdec492_rle_crop_3976579571_0.png temp/1758875363_4185575_1386052479_1ed2ec29d9d843c403d14ce8efadc8f8_rle_crop_3976579580_0.png temp/1758875363_4185575_1386052479_1ed2ec29d9d843c403d14ce8efadc8f8_rle_crop_3976579583_0.png temp/1758875363_4185575_1386052479_1ed2ec29d9d843c403d14ce8efadc8f8_rle_crop_3976579587_0.png temp/1758875363_4185575_1386049645_996ba35e66df1d993267147b532d6a76_rle_crop_3976579600_0.png temp/1758875363_4185575_1386049645_996ba35e66df1d993267147b532d6a76_rle_crop_3976579601_0.png temp/1758875363_4185575_1386049638_72aafc3860d22b74ed6f3a670941e651_rle_crop_3976579609_0.png temp/1758875363_4185575_1386049638_72aafc3860d22b74ed6f3a670941e651_rle_crop_3976579611_0.png temp/1758875363_4185575_1386049638_72aafc3860d22b74ed6f3a670941e651_rle_crop_3976579612_0.png temp/1758875363_4185575_1386049638_72aafc3860d22b74ed6f3a670941e651_rle_crop_3976579614_0.png temp/1758875363_4185575_1386049588_efe94e0f6eed9eebca9ccfcb33de407b_rle_crop_3976579626_0.png temp/1758875363_4185575_1386049588_efe94e0f6eed9eebca9ccfcb33de407b_rle_crop_3976579631_0.png temp/1758875363_4185575_1386049569_dd6a9a4752de3fd880b7ca8b4a28259e_rle_crop_3976579634_0.png temp/1758875363_4185575_1386049569_dd6a9a4752de3fd880b7ca8b4a28259e_rle_crop_3976579635_0.png temp/1758875363_4185575_1386049568_12ad5e14152cb315dc90a2094d5cb7b1_rle_crop_3976579644_0.png temp/1758875363_4185575_1386049568_12ad5e14152cb315dc90a2094d5cb7b1_rle_crop_3976579647_0.png temp/1758875363_4185575_1386049568_12ad5e14152cb315dc90a2094d5cb7b1_rle_crop_3976579651_0.png temp/1758875363_4185575_1386048577_a4455050ef07ca078b71ff28e85d9cd1_rle_crop_3976579661_0.png temp/1758875363_4185575_1386048569_0a4fd19d2357a5438009bd6c1fe1dd84_rle_crop_3976579672_0.png temp/1758875363_4185575_1386052877_1ec34129bf0d54ff7fdcfb15578a4ab7_rle_crop_3976579527_0.png temp/1758875363_4185575_1386052877_1ec34129bf0d54ff7fdcfb15578a4ab7_rle_crop_3976579528_0.png temp/1758875363_4185575_1386052877_1ec34129bf0d54ff7fdcfb15578a4ab7_rle_crop_3976579529_0.png temp/1758875363_4185575_1386052877_1ec34129bf0d54ff7fdcfb15578a4ab7_rle_crop_3976579531_0.png temp/1758875363_4185575_1386052877_1ec34129bf0d54ff7fdcfb15578a4ab7_rle_crop_3976579532_0.png temp/1758875363_4185575_1386052814_e15a80fcd0a96c572ecc7e562febea96_rle_crop_3976579540_0.png temp/1758875363_4185575_1386052814_e15a80fcd0a96c572ecc7e562febea96_rle_crop_3976579542_0.png temp/1758875363_4185575_1386052814_e15a80fcd0a96c572ecc7e562febea96_rle_crop_3976579544_0.png temp/1758875363_4185575_1386052750_91a1a2e8339b87a3f18054214b570365_rle_crop_3976579553_0.png temp/1758875363_4185575_1386052750_91a1a2e8339b87a3f18054214b570365_rle_crop_3976579555_0.png temp/1758875363_4185575_1386052750_91a1a2e8339b87a3f18054214b570365_rle_crop_3976579556_0.png temp/1758875363_4185575_1386052750_91a1a2e8339b87a3f18054214b570365_rle_crop_3976579557_0.png temp/1758875363_4185575_1386052651_16fab6254bfbf4d3edeb6432f962bb84_rle_crop_3976579560_0.png temp/1758875363_4185575_1386052479_1ed2ec29d9d843c403d14ce8efadc8f8_rle_crop_3976579581_0.png temp/1758875363_4185575_1386052479_1ed2ec29d9d843c403d14ce8efadc8f8_rle_crop_3976579586_0.png temp/1758875363_4185575_1386049645_996ba35e66df1d993267147b532d6a76_rle_crop_3976579592_0.png temp/1758875363_4185575_1386049645_996ba35e66df1d993267147b532d6a76_rle_crop_3976579593_0.png temp/1758875363_4185575_1386049645_996ba35e66df1d993267147b532d6a76_rle_crop_3976579594_0.png temp/1758875363_4185575_1386049645_996ba35e66df1d993267147b532d6a76_rle_crop_3976579596_0.png temp/1758875363_4185575_1386049645_996ba35e66df1d993267147b532d6a76_rle_crop_3976579597_0.png temp/1758875363_4185575_1386049638_72aafc3860d22b74ed6f3a670941e651_rle_crop_3976579604_0.png temp/1758875363_4185575_1386049638_72aafc3860d22b74ed6f3a670941e651_rle_crop_3976579607_0.png temp/1758875363_4185575_1386049638_72aafc3860d22b74ed6f3a670941e651_rle_crop_3976579608_0.png temp/1758875363_4185575_1386049619_ee6b0ab5fe6c9e96d9f464e9b1aea831_rle_crop_3976579617_0.png temp/1758875363_4185575_1386049619_ee6b0ab5fe6c9e96d9f464e9b1aea831_rle_crop_3976579619_0.png temp/1758875363_4185575_1386049619_ee6b0ab5fe6c9e96d9f464e9b1aea831_rle_crop_3976579620_0.png temp/1758875363_4185575_1386049619_ee6b0ab5fe6c9e96d9f464e9b1aea831_rle_crop_3976579621_0.png temp/1758875363_4185575_1386049588_efe94e0f6eed9eebca9ccfcb33de407b_rle_crop_3976579624_0.png temp/1758875363_4185575_1386049568_12ad5e14152cb315dc90a2094d5cb7b1_rle_crop_3976579645_0.png temp/1758875363_4185575_1386049568_12ad5e14152cb315dc90a2094d5cb7b1_rle_crop_3976579650_0.png temp/1758875363_4185575_1386048577_a4455050ef07ca078b71ff28e85d9cd1_rle_crop_3976579660_0.png temp/1758875363_4185575_1386048569_0a4fd19d2357a5438009bd6c1fe1dd84_rle_crop_3976579667_0.png temp/1758875363_4185575_1386048569_0a4fd19d2357a5438009bd6c1fe1dd84_rle_crop_3976579669_0.png temp/1758875363_4185575_1386052547_edf5b29219c6d609c6865c87dfdec492_rle_crop_3976579572_0.png 90it [00:00, 951.13it/s]2025-09-26 10:35:14.116750: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 10:35:14.148170: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 temp/1758875363_4185575_1386049569_dd6a9a4752de3fd880b7ca8b4a28259e_rle_crop_3976579636_0.png temp/1758875363_4185575_1386052651_16fab6254bfbf4d3edeb6432f962bb84_rle_crop_3976579563_0.png temp/1758875363_4185575_1386049588_efe94e0f6eed9eebca9ccfcb33de407b_rle_crop_3976579627_0.png Found 90 images belonging to 1 classes. begin to do the prediction : time used to do the prediction : 6.2679665088653564 (90,) (90, 10) (90, 1792) shape of features : (90, 1792) shape of new features : (1, 90, 1792) save descriptor for thcl : 3501 time to traite the descriptors : 0.5665829181671143 storage_type for insertDescriptorsMulti : 3 Missing photo l117 : -3976579526 Missing photo l117 : -3976579530 Missing photo l117 : -3976579541 Missing photo l117 : -3976579543 Missing photo l117 : -3976579561 Missing photo l117 : -3976579568 Missing photo l117 : -3976579582 Missing photo l117 : -3976579591 Missing photo l117 : -3976579595 Missing photo l117 : -3976579606 Missing photo l117 : -3976579610 Missing photo l117 : -3976579625 Missing photo l117 : -3976579632 Missing photo l117 : -3976579646 Missing photo l117 : -3976579658 Missing photo l117 : -3976579664 Missing photo l117 : -3976579559 Missing photo l117 : -3976579573 Missing photo l117 : -3976579575 Missing photo l117 : -3976579585 Missing photo l117 : -3976579623 Missing photo l117 : -3976579637 Missing photo l117 : -3976579639 Missing photo l117 : -3976579649 Missing photo l117 : -3976579654 Missing photo l117 : -3976579535 Missing photo l117 : -3976579536 Missing photo l117 : -3976579545 Missing photo l117 : -3976579546 Missing photo l117 : -3976579550 Missing photo l117 : -3976579552 Missing photo l117 : -3976579562 Missing photo l117 : -3976579567 Missing photo l117 : -3976579570 Missing photo l117 : -3976579571 Missing photo l117 : -3976579580 Missing photo l117 : -3976579583 Missing photo l117 : -3976579587 Missing photo l117 : -3976579600 Missing photo l117 : -3976579601 Missing photo l117 : -3976579609 Missing photo l117 : -3976579611 Missing photo l117 : -3976579612 Missing photo l117 : -3976579614 Missing photo l117 : -3976579626 Missing photo l117 : -3976579631 Missing photo l117 : -3976579634 Missing photo l117 : -3976579635 Missing photo l117 : -3976579644 Missing photo l117 : -3976579647 Missing photo l117 : -3976579651 Missing photo l117 : -3976579661 Missing photo l117 : -3976579672 Missing photo l117 : -3976579527 Missing photo l117 : -3976579528 Missing photo l117 : -3976579529 Missing photo l117 : -3976579531 Missing photo l117 : -3976579532 Missing photo l117 : -3976579540 Missing photo l117 : -3976579542 Missing photo l117 : -3976579544 Missing photo l117 : -3976579553 Missing photo l117 : -3976579555 Missing photo l117 : -3976579556 Missing photo l117 : -3976579557 Missing photo l117 : -3976579560 Missing photo l117 : -3976579581 Missing photo l117 : -3976579586 Missing photo l117 : -3976579592 Missing photo l117 : -3976579593 Missing photo l117 : -3976579594 Missing photo l117 : -3976579596 Missing photo l117 : -3976579597 Missing photo l117 : -3976579604 Missing photo l117 : -3976579607 Missing photo l117 : -3976579608 Missing photo l117 : -3976579617 Missing photo l117 : -3976579619 Missing photo l117 : -3976579620 Missing photo l117 : -3976579621 Missing photo l117 : -3976579624 Missing photo l117 : -3976579645 Missing photo l117 : -3976579650 Missing photo l117 : -3976579660 Missing photo l117 : -3976579667 Missing photo l117 : -3976579669 Missing photo l117 : -3976579572 Missing photo l117 : -3976579636 Missing photo l117 : -3976579563 Missing photo l117 : -3976579627 To insert : -3976579526 Missing photo l134 : -3976579526 To insert : -3976579530 Missing photo l134 : -3976579530 To insert : -3976579541 Missing photo l134 : -3976579541 To insert : -3976579543 Missing photo l134 : -3976579543 To insert : -3976579561 Missing photo l134 : -3976579561 To insert : -3976579568 Missing photo l134 : -3976579568 To insert : -3976579582 Missing photo l134 : -3976579582 To insert : -3976579591 Missing photo l134 : -3976579591 To insert : -3976579595 Missing photo l134 : -3976579595 To insert : -3976579606 Missing photo l134 : -3976579606 To insert : -3976579610 Missing photo l134 : -3976579610 To insert : -3976579625 Missing photo l134 : -3976579625 To insert : -3976579632 Missing photo l134 : -3976579632 To insert : -3976579646 Missing photo l134 : -3976579646 To insert : -3976579658 Missing photo l134 : -3976579658 To insert : -3976579664 Missing photo l134 : -3976579664 To insert : -3976579559 Missing photo l134 : -3976579559 To insert : -3976579573 Missing photo l134 : -3976579573 To insert : -3976579575 Missing photo l134 : -3976579575 To insert : -3976579585 Missing photo l134 : -3976579585 To insert : -3976579623 Missing photo l134 : -3976579623 To insert : -3976579637 Missing photo l134 : -3976579637 To insert : -3976579639 Missing photo l134 : -3976579639 To insert : -3976579649 Missing photo l134 : -3976579649 To insert : -3976579654 Missing photo l134 : -3976579654 To insert : -3976579535 Missing photo l134 : -3976579535 To insert : -3976579536 Missing photo l134 : -3976579536 To insert : -3976579545 Missing photo l134 : -3976579545 To insert : -3976579546 Missing photo l134 : -3976579546 To insert : -3976579550 Missing photo l134 : -3976579550 To insert : -3976579552 Missing photo l134 : -3976579552 To insert : -3976579562 Missing photo l134 : -3976579562 To insert : -3976579567 Missing photo l134 : -3976579567 To insert : -3976579570 Missing photo l134 : -3976579570 To insert : -3976579571 Missing photo l134 : -3976579571 To insert : -3976579580 Missing photo l134 : -3976579580 To insert : -3976579583 Missing photo l134 : -3976579583 To insert : -3976579587 Missing photo l134 : -3976579587 To insert : -3976579600 Missing photo l134 : -3976579600 To insert : -3976579601 Missing photo l134 : -3976579601 To insert : -3976579609 Missing photo l134 : -3976579609 To insert : -3976579611 Missing photo l134 : -3976579611 To insert : -3976579612 Missing photo l134 : -3976579612 To insert : -3976579614 Missing photo l134 : -3976579614 To insert : -3976579626 Missing photo l134 : -3976579626 To insert : -3976579631 Missing photo l134 : -3976579631 To insert : -3976579634 Missing photo l134 : -3976579634 To insert : -3976579635 Missing photo l134 : -3976579635 To insert : -3976579644 Missing photo l134 : -3976579644 To insert : -3976579647 Missing photo l134 : -3976579647 To insert : -3976579651 Missing photo l134 : -3976579651 To insert : -3976579661 Missing photo l134 : -3976579661 To insert : -3976579672 Missing photo l134 : -3976579672 To insert : -3976579527 Missing photo l134 : -3976579527 To insert : -3976579528 Missing photo l134 : -3976579528 To insert : -3976579529 Missing photo l134 : -3976579529 To insert : -3976579531 Missing photo l134 : -3976579531 To insert : -3976579532 Missing photo l134 : -3976579532 To insert : -3976579540 Missing photo l134 : -3976579540 To insert : -3976579542 Missing photo l134 : -3976579542 To insert : -3976579544 Missing photo l134 : -3976579544 To insert : -3976579553 Missing photo l134 : -3976579553 To insert : -3976579555 Missing photo l134 : -3976579555 To insert : -3976579556 Missing photo l134 : -3976579556 To insert : -3976579557 Missing photo l134 : -3976579557 To insert : -3976579560 Missing photo l134 : -3976579560 To insert : -3976579581 Missing photo l134 : -3976579581 To insert : -3976579586 Missing photo l134 : -3976579586 To insert : -3976579592 Missing photo l134 : -3976579592 To insert : -3976579593 Missing photo l134 : -3976579593 To insert : -3976579594 Missing photo l134 : -3976579594 To insert : -3976579596 Missing photo l134 : -3976579596 To insert : -3976579597 Missing photo l134 : -3976579597 To insert : -3976579604 Missing photo l134 : -3976579604 To insert : -3976579607 Missing photo l134 : -3976579607 To insert : -3976579608 Missing photo l134 : -3976579608 To insert : -3976579617 Missing photo l134 : -3976579617 To insert : -3976579619 Missing photo l134 : -3976579619 To insert : -3976579620 Missing photo l134 : -3976579620 To insert : -3976579621 Missing photo l134 : -3976579621 To insert : -3976579624 Missing photo l134 : -3976579624 To insert : -3976579645 Missing photo l134 : -3976579645 To insert : -3976579650 Missing photo l134 : -3976579650 To insert : -3976579660 Missing photo l134 : -3976579660 To insert : -3976579667 Missing photo l134 : -3976579667 To insert : -3976579669 Missing photo l134 : -3976579669 To insert : -3976579572 Missing photo l134 : -3976579572 To insert : -3976579636 Missing photo l134 : -3976579636 To insert : -3976579563 Missing photo l134 : -3976579563 To insert : -3976579627 Missing photo l134 : -3976579627 time to insert the descriptors : 20.480833292007446 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : tfhub_classification2 we use saveGeneral [1386052877, 1386052814, 1386052750, 1386052651, 1386052547, 1386052479, 1386049645, 1386049638, 1386049619, 1386049588, 1386049569, 1386049568, 1386048608, 1386048577, 1386048569] Looping around the photos to save general results len do output : 90 /-3976579526Didn't retrieve data . /-3976579530Didn't retrieve data . /-3976579541Didn't retrieve data . /-3976579543Didn't retrieve data . /-3976579561Didn't retrieve data . /-3976579568Didn't retrieve data . /-3976579582Didn't retrieve data . /-3976579591Didn't retrieve data . /-3976579595Didn't retrieve data . /-3976579606Didn't retrieve data . /-3976579610Didn't retrieve data . /-3976579625Didn't retrieve data . /-3976579632Didn't retrieve data . /-3976579646Didn't retrieve data . /-3976579658Didn't retrieve data . /-3976579664Didn't retrieve data . /-3976579559Didn't retrieve data . /-3976579573Didn't retrieve data . /-3976579575Didn't retrieve data . /-3976579585Didn't retrieve data . /-3976579623Didn't retrieve data . /-3976579637Didn't retrieve data . /-3976579639Didn't retrieve data . /-3976579649Didn't retrieve data . /-3976579654Didn't retrieve data . /-3976579535Didn't retrieve data . /-3976579536Didn't retrieve data . /-3976579545Didn't retrieve data . /-3976579546Didn't retrieve data . /-3976579550Didn't retrieve data . /-3976579552Didn't retrieve data . /-3976579562Didn't retrieve data . /-3976579567Didn't retrieve data . /-3976579570Didn't retrieve data . /-3976579571Didn't retrieve data . /-3976579580Didn't retrieve data . /-3976579583Didn't retrieve data . /-3976579587Didn't retrieve data . /-3976579600Didn't retrieve data . /-3976579601Didn't retrieve data . /-3976579609Didn't retrieve data . /-3976579611Didn't retrieve data . /-3976579612Didn't retrieve data . /-3976579614Didn't retrieve data . /-3976579626Didn't retrieve data . /-3976579631Didn't retrieve data . /-3976579634Didn't retrieve data . /-3976579635Didn't retrieve data . /-3976579644Didn't retrieve data . /-3976579647Didn't retrieve data . /-3976579651Didn't retrieve data . /-3976579661Didn't retrieve data . /-3976579672Didn't retrieve data . /-3976579527Didn't retrieve data . /-3976579528Didn't retrieve data . /-3976579529Didn't retrieve data . /-3976579531Didn't retrieve data . /-3976579532Didn't retrieve data . /-3976579540Didn't retrieve data . /-3976579542Didn't retrieve data . /-3976579544Didn't retrieve data . /-3976579553Didn't retrieve data . /-3976579555Didn't retrieve data . /-3976579556Didn't retrieve data . /-3976579557Didn't retrieve data . /-3976579560Didn't retrieve data . /-3976579581Didn't retrieve data . /-3976579586Didn't retrieve data . /-3976579592Didn't retrieve data . /-3976579593Didn't retrieve data . /-3976579594Didn't retrieve data . /-3976579596Didn't retrieve data . /-3976579597Didn't retrieve data . /-3976579604Didn't retrieve data . /-3976579607Didn't retrieve data . /-3976579608Didn't retrieve data . /-3976579617Didn't retrieve data . /-3976579619Didn't retrieve data . /-3976579620Didn't retrieve data . /-3976579621Didn't retrieve data . /-3976579624Didn't retrieve data . /-3976579645Didn't retrieve data . /-3976579650Didn't retrieve data . /-3976579660Didn't retrieve data . /-3976579667Didn't retrieve data . /-3976579669Didn't retrieve data . /-3976579572Didn't retrieve data . /-3976579636Didn't retrieve data . /-3976579563Didn't retrieve data . /-3976579627Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052877', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052814', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052750', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052651', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052547', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052479', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049645', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049638', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049619', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049588', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049569', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049568', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048608', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048577', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048569', None, None, None, None, None, '3784106') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 105 time used for this insertion : 0.027910470962524414 save_final save missing photos in datou_result : time spend for datou_step_exec : 54.82897663116455 time spend to save output : 0.03756141662597656 total time spend for step 6 : 54.86653804779053 step7:merge_mask_thcl_custom Fri Sep 26 10:35:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 150 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1386052877, 1386052814, 1386052750, 1386052651, 1386052547, 1386052479, 1386049645, 1386049638, 1386049619, 1386049588, 1386049569, 1386049568, 1386048608, 1386048577, 1386048569] Looping around the photos to save general results len do output : 15 /1386052877Didn't retrieve data .Didn't retrieve data . /1386052814Didn't retrieve data .Didn't retrieve data . /1386052750Didn't retrieve data .Didn't retrieve data . /1386052651Didn't retrieve data .Didn't retrieve data . /1386052547Didn't retrieve data .Didn't retrieve data . /1386052479Didn't retrieve data .Didn't retrieve data . /1386049645Didn't retrieve data .Didn't retrieve data . /1386049638Didn't retrieve data .Didn't retrieve data . /1386049619Didn't retrieve data .Didn't retrieve data . /1386049588Didn't retrieve data .Didn't retrieve data . /1386049569Didn't retrieve data .Didn't retrieve data . /1386049568Didn't retrieve data .Didn't retrieve data . /1386048608Didn't retrieve data .Didn't retrieve data . /1386048577Didn't retrieve data .Didn't retrieve data . /1386048569Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052877', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052814', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052750', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052651', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052547', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052479', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049645', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049638', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049619', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049588', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049569', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049568', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048608', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048577', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048569', None, None, None, None, None, '3784106') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 45 time used for this insertion : 0.013040781021118164 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.06701970100402832 time spend to save output : 0.013797998428344727 total time spend for step 7 : 0.08081769943237305 step8:rle_unique_nms_with_priority Fri Sep 26 10:35:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 14 nb_hashtags : 4 time to prepare the origin masks : 17.8013277053833 time for calcul the mask position with numpy : 0.33541107177734375 nb_pixel_total : 9258878 time to create 1 rle with new method : 1.9129834175109863 time for calcul the mask position with numpy : 0.05880379676818848 nb_pixel_total : 14581 time to create 1 rle with old method : 0.016541481018066406 time for calcul the mask position with numpy : 0.05822038650512695 nb_pixel_total : 19373 time to create 1 rle with old method : 0.022897958755493164 time for calcul the mask position with numpy : 0.04272723197937012 nb_pixel_total : 384781 time to create 1 rle with new method : 0.6840000152587891 time for calcul the mask position with numpy : 0.04145383834838867 nb_pixel_total : 401222 time to create 1 rle with new method : 0.584991455078125 time for calcul the mask position with numpy : 0.0424041748046875 nb_pixel_total : 87409 time to create 1 rle with old method : 0.09961223602294922 time for calcul the mask position with numpy : 0.05373072624206543 nb_pixel_total : 439851 time to create 1 rle with new method : 0.7472524642944336 time for calcul the mask position with numpy : 0.06263375282287598 nb_pixel_total : 30655 time to create 1 rle with old method : 0.03768658638000488 time for calcul the mask position with numpy : 0.10163497924804688 nb_pixel_total : 1127147 time to create 1 rle with new method : 1.0612578392028809 time for calcul the mask position with numpy : 0.058075904846191406 nb_pixel_total : 11538 time to create 1 rle with old method : 0.012786149978637695 time for calcul the mask position with numpy : 0.05825662612915039 nb_pixel_total : 27698 time to create 1 rle with old method : 0.030612468719482422 time for calcul the mask position with numpy : 0.05686187744140625 nb_pixel_total : 395613 time to create 1 rle with new method : 2.025219440460205 time for calcul the mask position with numpy : 0.08265280723571777 nb_pixel_total : 64116 time to create 1 rle with old method : 0.07385396957397461 time for calcul the mask position with numpy : 0.07382678985595703 nb_pixel_total : 47904 time to create 1 rle with old method : 0.05671954154968262 time for calcul the mask position with numpy : 0.05250859260559082 nb_pixel_total : 19474 time to create 1 rle with old method : 0.02490520477294922 create new chi : 8.854237079620361 time to delete rle : 0.018939495086669922 batch 1 Loaded 15 chid ids of type : 3726 Number RLEs to save : 19745 TO DO : save crop sub photo not yet done ! save time : 1.2838199138641357 nb_obj : 11 nb_hashtags : 5 time to prepare the origin masks : 16.025683164596558 time for calcul the mask position with numpy : 0.44701433181762695 nb_pixel_total : 10572347 time to create 1 rle with new method : 0.6175787448883057 time for calcul the mask position with numpy : 0.07567572593688965 nb_pixel_total : 118742 time to create 1 rle with old method : 0.1385965347290039 time for calcul the mask position with numpy : 0.40080952644348145 nb_pixel_total : 225933 time to create 1 rle with new method : 0.9947962760925293 time for calcul the mask position with numpy : 0.06745743751525879 nb_pixel_total : 56902 time to create 1 rle with old method : 0.06765890121459961 time for calcul the mask position with numpy : 0.06646394729614258 nb_pixel_total : 8897 time to create 1 rle with old method : 0.011487245559692383 time for calcul the mask position with numpy : 0.049149274826049805 nb_pixel_total : 119640 time to create 1 rle with old method : 0.14210867881774902 time for calcul the mask position with numpy : 0.03620624542236328 nb_pixel_total : 122762 time to create 1 rle with old method : 0.18653130531311035 time for calcul the mask position with numpy : 0.12055826187133789 nb_pixel_total : 890431 time to create 1 rle with new method : 0.6306593418121338 time for calcul the mask position with numpy : 0.040454864501953125 nb_pixel_total : 5963 time to create 1 rle with old method : 0.0067675113677978516 time for calcul the mask position with numpy : 0.04286766052246094 nb_pixel_total : 8474 time to create 1 rle with old method : 0.009850263595581055 time for calcul the mask position with numpy : 0.037911176681518555 nb_pixel_total : 110908 time to create 1 rle with old method : 0.12308549880981445 time for calcul the mask position with numpy : 0.09397435188293457 nb_pixel_total : 89241 time to create 1 rle with old method : 0.09918498992919922 create new chi : 4.661773681640625 time to delete rle : 0.0018429756164550781 batch 1 Loaded 12 chid ids of type : 3726 Number RLEs to save : 18433 TO DO : save crop sub photo not yet done ! save time : 1.451348066329956 nb_obj : 5 nb_hashtags : 3 time to prepare the origin masks : 7.850576400756836 time for calcul the mask position with numpy : 0.7081036567687988 nb_pixel_total : 10687291 time to create 1 rle with new method : 0.897822380065918 time for calcul the mask position with numpy : 0.04651927947998047 nb_pixel_total : 364794 time to create 1 rle with new method : 0.6153903007507324 time for calcul the mask position with numpy : 0.046280622482299805 nb_pixel_total : 26086 time to create 1 rle with old method : 0.06759881973266602 time for calcul the mask position with numpy : 0.07444906234741211 nb_pixel_total : 75710 time to create 1 rle with old method : 0.08295321464538574 time for calcul the mask position with numpy : 0.06280183792114258 nb_pixel_total : 84874 time to create 1 rle with old method : 0.09251904487609863 time for calcul the mask position with numpy : 0.07374405860900879 nb_pixel_total : 1091485 time to create 1 rle with new method : 1.0015146732330322 create new chi : 3.916276454925537 time to delete rle : 0.001577615737915039 batch 1 Loaded 6 chid ids of type : 3726 Number RLEs to save : 10201 TO DO : save crop sub photo not yet done ! save time : 0.6120481491088867 nb_obj : 7 nb_hashtags : 5 time to prepare the origin masks : 8.631651639938354 time for calcul the mask position with numpy : 0.47678542137145996 nb_pixel_total : 10891101 time to create 1 rle with new method : 0.6591098308563232 time for calcul the mask position with numpy : 0.08093547821044922 nb_pixel_total : 8557 time to create 1 rle with old method : 0.0100555419921875 time for calcul the mask position with numpy : 0.05821800231933594 nb_pixel_total : 141888 time to create 1 rle with old method : 0.14856863021850586 time for calcul the mask position with numpy : 0.06625151634216309 nb_pixel_total : 66356 time to create 1 rle with old method : 0.07093262672424316 time for calcul the mask position with numpy : 0.08783817291259766 nb_pixel_total : 37121 time to create 1 rle with old method : 0.05209541320800781 time for calcul the mask position with numpy : 0.06719470024108887 nb_pixel_total : 168405 time to create 1 rle with new method : 1.1799705028533936 time for calcul the mask position with numpy : 0.07763481140136719 nb_pixel_total : 975823 time to create 1 rle with new method : 0.6787188053131104 time for calcul the mask position with numpy : 0.0775003433227539 nb_pixel_total : 40989 time to create 1 rle with old method : 0.16605448722839355 create new chi : 4.1559975147247314 time to delete rle : 0.0016734600067138672 batch 1 Loaded 8 chid ids of type : 3726 Number RLEs to save : 11189 TO DO : save crop sub photo not yet done ! save time : 0.6579017639160156 nb_obj : 9 nb_hashtags : 3 time to prepare the origin masks : 13.30306887626648 time for calcul the mask position with numpy : 0.4465484619140625 nb_pixel_total : 10357879 time to create 1 rle with new method : 1.2692008018493652 time for calcul the mask position with numpy : 0.07419395446777344 nb_pixel_total : 125498 time to create 1 rle with old method : 0.13609719276428223 time for calcul the mask position with numpy : 0.07216548919677734 nb_pixel_total : 4825 time to create 1 rle with old method : 0.0055980682373046875 time for calcul the mask position with numpy : 0.0917813777923584 nb_pixel_total : 658127 time to create 1 rle with new method : 0.8132309913635254 time for calcul the mask position with numpy : 0.07517457008361816 nb_pixel_total : 404367 time to create 1 rle with new method : 0.91554856300354 time for calcul the mask position with numpy : 0.057082414627075195 nb_pixel_total : 190398 time to create 1 rle with new method : 0.6801595687866211 time for calcul the mask position with numpy : 0.04250311851501465 nb_pixel_total : 120252 time to create 1 rle with old method : 0.1443781852722168 time for calcul the mask position with numpy : 0.043611764907836914 nb_pixel_total : 427814 time to create 1 rle with new method : 0.6909291744232178 time for calcul the mask position with numpy : 0.04295229911804199 nb_pixel_total : 24328 time to create 1 rle with old method : 0.027512550354003906 time for calcul the mask position with numpy : 0.042528629302978516 nb_pixel_total : 16752 time to create 1 rle with old method : 0.0200345516204834 create new chi : 5.962996482849121 time to delete rle : 0.0020973682403564453 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 13948 TO DO : save crop sub photo not yet done ! save time : 0.7872805595397949 nb_obj : 9 nb_hashtags : 4 time to prepare the origin masks : 14.496801614761353 time for calcul the mask position with numpy : 0.47369384765625 nb_pixel_total : 11485292 time to create 1 rle with new method : 0.734605073928833 time for calcul the mask position with numpy : 0.06723952293395996 nb_pixel_total : 95914 time to create 1 rle with old method : 0.11082124710083008 time for calcul the mask position with numpy : 0.06684303283691406 nb_pixel_total : 409769 time to create 1 rle with new method : 0.7814407348632812 time for calcul the mask position with numpy : 0.041383981704711914 nb_pixel_total : 15881 time to create 1 rle with old method : 0.020689725875854492 time for calcul the mask position with numpy : 0.04409432411193848 nb_pixel_total : 50285 time to create 1 rle with old method : 0.06831884384155273 time for calcul the mask position with numpy : 0.03979229927062988 nb_pixel_total : 20482 time to create 1 rle with old method : 0.023339271545410156 time for calcul the mask position with numpy : 0.03988909721374512 nb_pixel_total : 28106 time to create 1 rle with old method : 0.03192591667175293 time for calcul the mask position with numpy : 0.04246258735656738 nb_pixel_total : 60792 time to create 1 rle with old method : 0.11215448379516602 time for calcul the mask position with numpy : 0.0397486686706543 nb_pixel_total : 12086 time to create 1 rle with old method : 0.014116764068603516 time for calcul the mask position with numpy : 0.08501577377319336 nb_pixel_total : 151633 time to create 1 rle with new method : 0.8990797996520996 create new chi : 3.8906872272491455 time to delete rle : 0.0011348724365234375 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 9232 TO DO : save crop sub photo not yet done ! save time : 0.6384198665618896 nb_obj : 13 nb_hashtags : 4 time to prepare the origin masks : 16.44739842414856 time for calcul the mask position with numpy : 0.3490445613861084 nb_pixel_total : 9347723 time to create 1 rle with new method : 0.6733918190002441 time for calcul the mask position with numpy : 0.03466176986694336 nb_pixel_total : 14583 time to create 1 rle with old method : 0.015926599502563477 time for calcul the mask position with numpy : 0.04086494445800781 nb_pixel_total : 19655 time to create 1 rle with old method : 0.0226287841796875 time for calcul the mask position with numpy : 0.036675453186035156 nb_pixel_total : 384411 time to create 1 rle with new method : 0.9890239238739014 time for calcul the mask position with numpy : 0.041385650634765625 nb_pixel_total : 400363 time to create 1 rle with new method : 0.8236217498779297 time for calcul the mask position with numpy : 0.0409083366394043 nb_pixel_total : 439868 time to create 1 rle with new method : 0.7850632667541504 time for calcul the mask position with numpy : 0.038588762283325195 nb_pixel_total : 30640 time to create 1 rle with old method : 0.034139156341552734 time for calcul the mask position with numpy : 0.03875231742858887 nb_pixel_total : 11535 time to create 1 rle with old method : 0.013221979141235352 time for calcul the mask position with numpy : 0.04675889015197754 nb_pixel_total : 1127408 time to create 1 rle with new method : 0.6637465953826904 time for calcul the mask position with numpy : 0.036232709884643555 nb_pixel_total : 27676 time to create 1 rle with old method : 0.02940058708190918 time for calcul the mask position with numpy : 0.041879892349243164 nb_pixel_total : 394972 time to create 1 rle with new method : 0.6389424800872803 time for calcul the mask position with numpy : 0.04289412498474121 nb_pixel_total : 64185 time to create 1 rle with old method : 0.09092879295349121 time for calcul the mask position with numpy : 0.043851375579833984 nb_pixel_total : 47786 time to create 1 rle with old method : 0.05518984794616699 time for calcul the mask position with numpy : 0.03930783271789551 nb_pixel_total : 19435 time to create 1 rle with old method : 0.022167444229125977 create new chi : 5.991692066192627 time to delete rle : 0.0019183158874511719 batch 1 Loaded 14 chid ids of type : 3726 Number RLEs to save : 19082 TO DO : save crop sub photo not yet done ! save time : 1.1349608898162842 nb_obj : 11 nb_hashtags : 5 time to prepare the origin masks : 14.43940258026123 time for calcul the mask position with numpy : 0.6823797225952148 nb_pixel_total : 10369300 time to create 1 rle with new method : 1.1551508903503418 time for calcul the mask position with numpy : 0.07778310775756836 nb_pixel_total : 118768 time to create 1 rle with old method : 0.13966989517211914 time for calcul the mask position with numpy : 0.06618833541870117 nb_pixel_total : 225740 time to create 1 rle with new method : 0.7276337146759033 time for calcul the mask position with numpy : 0.06400132179260254 nb_pixel_total : 56668 time to create 1 rle with old method : 0.07161593437194824 time for calcul the mask position with numpy : 0.03534102439880371 nb_pixel_total : 8922 time to create 1 rle with old method : 0.009864091873168945 time for calcul the mask position with numpy : 0.03736281394958496 nb_pixel_total : 124281 time to create 1 rle with old method : 0.1308746337890625 time for calcul the mask position with numpy : 0.04129815101623535 nb_pixel_total : 8179 time to create 1 rle with old method : 0.010989665985107422 time for calcul the mask position with numpy : 0.04026389122009277 nb_pixel_total : 122630 time to create 1 rle with old method : 0.13493943214416504 time for calcul the mask position with numpy : 0.03723645210266113 nb_pixel_total : 5961 time to create 1 rle with old method : 0.007196187973022461 time for calcul the mask position with numpy : 0.04043936729431152 nb_pixel_total : 88276 time to create 1 rle with old method : 0.09830379486083984 time for calcul the mask position with numpy : 0.03744387626647949 nb_pixel_total : 111060 time to create 1 rle with old method : 0.12340998649597168 time for calcul the mask position with numpy : 0.050614118576049805 nb_pixel_total : 1090455 time to create 1 rle with new method : 0.5858662128448486 create new chi : 4.654994487762451 time to delete rle : 0.0026237964630126953 batch 1 Loaded 12 chid ids of type : 3726 Number RLEs to save : 17679 TO DO : save crop sub photo not yet done ! save time : 1.0744993686676025 nb_obj : 5 nb_hashtags : 3 time to prepare the origin masks : 8.546139001846313 time for calcul the mask position with numpy : 0.9700102806091309 nb_pixel_total : 10687765 time to create 1 rle with new method : 0.661862850189209 time for calcul the mask position with numpy : 0.06777572631835938 nb_pixel_total : 364701 time to create 1 rle with new method : 1.0909960269927979 time for calcul the mask position with numpy : 0.07404327392578125 nb_pixel_total : 26103 time to create 1 rle with old method : 0.03369450569152832 time for calcul the mask position with numpy : 0.06203579902648926 nb_pixel_total : 75539 time to create 1 rle with old method : 0.09032917022705078 time for calcul the mask position with numpy : 0.03862142562866211 nb_pixel_total : 84856 time to create 1 rle with old method : 0.09653401374816895 time for calcul the mask position with numpy : 0.04882097244262695 nb_pixel_total : 1091276 time to create 1 rle with new method : 0.6448013782501221 create new chi : 4.027151823043823 time to delete rle : 0.001607656478881836 batch 1 Loaded 6 chid ids of type : 3726 Number RLEs to save : 10197 TO DO : save crop sub photo not yet done ! save time : 0.6055996417999268 nb_obj : 7 nb_hashtags : 5 time to prepare the origin masks : 8.479857206344604 time for calcul the mask position with numpy : 0.48363494873046875 nb_pixel_total : 10894914 time to create 1 rle with new method : 0.7084805965423584 time for calcul the mask position with numpy : 0.04047703742980957 nb_pixel_total : 8667 time to create 1 rle with old method : 0.010179281234741211 time for calcul the mask position with numpy : 0.04778313636779785 nb_pixel_total : 141932 time to create 1 rle with old method : 0.18105363845825195 time for calcul the mask position with numpy : 0.05033445358276367 nb_pixel_total : 66333 time to create 1 rle with old method : 0.07545995712280273 time for calcul the mask position with numpy : 0.05069303512573242 nb_pixel_total : 37131 time to create 1 rle with old method : 0.04924368858337402 time for calcul the mask position with numpy : 0.05504465103149414 nb_pixel_total : 168389 time to create 1 rle with new method : 0.9768857955932617 time for calcul the mask position with numpy : 0.08105897903442383 nb_pixel_total : 971732 time to create 1 rle with new method : 0.9058310985565186 time for calcul the mask position with numpy : 0.08714890480041504 nb_pixel_total : 41142 time to create 1 rle with old method : 0.044888973236083984 create new chi : 4.029559135437012 time to delete rle : 0.0013794898986816406 batch 1 Loaded 8 chid ids of type : 3726 Number RLEs to save : 11179 TO DO : save crop sub photo not yet done ! save time : 0.6400179862976074 nb_obj : 9 nb_hashtags : 3 time to prepare the origin masks : 12.048482894897461 time for calcul the mask position with numpy : 0.9651317596435547 nb_pixel_total : 10356759 time to create 1 rle with new method : 0.8261947631835938 time for calcul the mask position with numpy : 0.07188630104064941 nb_pixel_total : 125541 time to create 1 rle with old method : 0.14030003547668457 time for calcul the mask position with numpy : 0.06615018844604492 nb_pixel_total : 4813 time to create 1 rle with old method : 0.008038043975830078 time for calcul the mask position with numpy : 0.07707405090332031 nb_pixel_total : 657859 time to create 1 rle with new method : 1.066671371459961 time for calcul the mask position with numpy : 0.062383174896240234 nb_pixel_total : 405092 time to create 1 rle with new method : 0.999638557434082 time for calcul the mask position with numpy : 0.04812264442443848 nb_pixel_total : 190532 time to create 1 rle with new method : 1.0141408443450928 time for calcul the mask position with numpy : 0.06719112396240234 nb_pixel_total : 120601 time to create 1 rle with old method : 0.13933372497558594 time for calcul the mask position with numpy : 0.06212162971496582 nb_pixel_total : 427948 time to create 1 rle with new method : 1.1243541240692139 time for calcul the mask position with numpy : 0.06278109550476074 nb_pixel_total : 24343 time to create 1 rle with old method : 0.02797412872314453 time for calcul the mask position with numpy : 0.05624127388000488 nb_pixel_total : 16752 time to create 1 rle with old method : 0.019465923309326172 create new chi : 7.152863502502441 time to delete rle : 0.002227306365966797 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 13922 TO DO : save crop sub photo not yet done ! save time : 0.9481091499328613 nb_obj : 9 nb_hashtags : 4 time to prepare the origin masks : 11.260415315628052 time for calcul the mask position with numpy : 1.2829413414001465 nb_pixel_total : 11484942 time to create 1 rle with new method : 2.030982494354248 time for calcul the mask position with numpy : 0.06888747215270996 nb_pixel_total : 95940 time to create 1 rle with old method : 0.10726380348205566 time for calcul the mask position with numpy : 0.06802988052368164 nb_pixel_total : 410036 time to create 1 rle with new method : 1.1714308261871338 time for calcul the mask position with numpy : 0.06558609008789062 nb_pixel_total : 15890 time to create 1 rle with old method : 0.018497228622436523 time for calcul the mask position with numpy : 0.06193375587463379 nb_pixel_total : 50285 time to create 1 rle with old method : 0.06022453308105469 time for calcul the mask position with numpy : 0.04265189170837402 nb_pixel_total : 20482 time to create 1 rle with old method : 0.02774214744567871 time for calcul the mask position with numpy : 0.04621171951293945 nb_pixel_total : 28088 time to create 1 rle with old method : 0.039563894271850586 time for calcul the mask position with numpy : 0.051294803619384766 nb_pixel_total : 60847 time to create 1 rle with old method : 0.06899118423461914 time for calcul the mask position with numpy : 0.04476785659790039 nb_pixel_total : 12079 time to create 1 rle with old method : 0.013827323913574219 time for calcul the mask position with numpy : 0.0595090389251709 nb_pixel_total : 151651 time to create 1 rle with new method : 0.6908941268920898 create new chi : 6.172697305679321 time to delete rle : 0.0011730194091796875 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 9236 TO DO : save crop sub photo not yet done ! save time : 0.5498301982879639 nb_obj : 5 nb_hashtags : 1 time to prepare the origin masks : 8.642513751983643 time for calcul the mask position with numpy : 1.3173751831054688 nb_pixel_total : 10212275 time to create 1 rle with new method : 1.1970837116241455 time for calcul the mask position with numpy : 0.0362391471862793 nb_pixel_total : 19468 time to create 1 rle with old method : 0.022269248962402344 time for calcul the mask position with numpy : 0.04810380935668945 nb_pixel_total : 517943 time to create 1 rle with new method : 1.3851006031036377 time for calcul the mask position with numpy : 0.1336653232574463 nb_pixel_total : 318463 time to create 1 rle with new method : 1.0511574745178223 time for calcul the mask position with numpy : 0.03645682334899902 nb_pixel_total : 18500 time to create 1 rle with old method : 0.02079629898071289 time for calcul the mask position with numpy : 0.044368743896484375 nb_pixel_total : 1243591 time to create 1 rle with new method : 1.1211934089660645 create new chi : 6.599634408950806 time to delete rle : 0.0015513896942138672 batch 1 Loaded 6 chid ids of type : 3726 Number RLEs to save : 14870 TO DO : save crop sub photo not yet done ! save time : 0.8711082935333252 nb_obj : 8 nb_hashtags : 4 time to prepare the origin masks : 12.457642793655396 time for calcul the mask position with numpy : 0.4407196044921875 nb_pixel_total : 7907544 time to create 1 rle with new method : 0.9643521308898926 time for calcul the mask position with numpy : 0.04273819923400879 nb_pixel_total : 1223159 time to create 1 rle with new method : 1.346919298171997 time for calcul the mask position with numpy : 0.04125642776489258 nb_pixel_total : 170168 time to create 1 rle with new method : 1.2468369007110596 time for calcul the mask position with numpy : 0.046447038650512695 nb_pixel_total : 159397 time to create 1 rle with new method : 1.2730193138122559 time for calcul the mask position with numpy : 0.09058594703674316 nb_pixel_total : 836666 time to create 1 rle with new method : 1.1596360206604004 time for calcul the mask position with numpy : 0.0720677375793457 nb_pixel_total : 8182 time to create 1 rle with old method : 0.015392780303955078 time for calcul the mask position with numpy : 0.09606218338012695 nb_pixel_total : 963 time to create 1 rle with old method : 0.0017006397247314453 time for calcul the mask position with numpy : 0.07106447219848633 nb_pixel_total : 85640 time to create 1 rle with old method : 0.09820747375488281 time for calcul the mask position with numpy : 0.07513689994812012 nb_pixel_total : 1938521 time to create 1 rle with new method : 1.0742173194885254 create new chi : 8.433095932006836 time to delete rle : 0.0026001930236816406 batch 1 Loaded 9 chid ids of type : 3726 Number RLEs to save : 18633 TO DO : save crop sub photo not yet done ! save time : 1.2556250095367432 nb_obj : 6 nb_hashtags : 4 time to prepare the origin masks : 9.38962984085083 time for calcul the mask position with numpy : 0.8261525630950928 nb_pixel_total : 9353061 time to create 1 rle with new method : 1.0469284057617188 time for calcul the mask position with numpy : 0.03783106803894043 nb_pixel_total : 156229 time to create 1 rle with new method : 1.8015599250793457 time for calcul the mask position with numpy : 0.06478214263916016 nb_pixel_total : 8826 time to create 1 rle with old method : 0.016065597534179688 time for calcul the mask position with numpy : 0.03632926940917969 nb_pixel_total : 68749 time to create 1 rle with old method : 0.0838479995727539 time for calcul the mask position with numpy : 0.12009882926940918 nb_pixel_total : 36898 time to create 1 rle with old method : 0.04121804237365723 time for calcul the mask position with numpy : 0.037157297134399414 nb_pixel_total : 5957 time to create 1 rle with old method : 0.00944375991821289 time for calcul the mask position with numpy : 0.16101598739624023 nb_pixel_total : 2700520 time to create 1 rle with new method : 1.0634286403656006 create new chi : 5.49230170249939 time to delete rle : 0.001409292221069336 batch 1 Loaded 7 chid ids of type : 3726 Number RLEs to save : 10864 TO DO : save crop sub photo not yet done ! save time : 0.8107683658599854 map_output_result : {1386052877: (0.0, 'Should be the crop_list due to order', 0.0), 1386052814: (0.0, 'Should be the crop_list due to order', 0.0), 1386052750: (0.0, 'Should be the crop_list due to order', 0.0), 1386052651: (0.0, 'Should be the crop_list due to order', 0.0), 1386052547: (0.0, 'Should be the crop_list due to order', 0.0), 1386052479: (0.0, 'Should be the crop_list due to order', 0.0), 1386049645: (0.0, 'Should be the crop_list due to order', 0.0), 1386049638: (0.0, 'Should be the crop_list due to order', 0.0), 1386049619: (0.0, 'Should be the crop_list due to order', 0.0), 1386049588: (0.0, 'Should be the crop_list due to order', 0.0), 1386049569: (0.0, 'Should be the crop_list due to order', 0.0), 1386049568: (0.0, 'Should be the crop_list due to order', 0.0), 1386048608: (0.0, 'Should be the crop_list due to order', 0.0), 1386048577: (0.0, 'Should be the crop_list due to order', 0.0), 1386048569: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1386052877, 1386052814, 1386052750, 1386052651, 1386052547, 1386052479, 1386049645, 1386049638, 1386049619, 1386049588, 1386049569, 1386049568, 1386048608, 1386048577, 1386048569] Looping around the photos to save general results len do output : 15 /1386052877.Didn't retrieve data . /1386052814.Didn't retrieve data . /1386052750.Didn't retrieve data . /1386052651.Didn't retrieve data . /1386052547.Didn't retrieve data . /1386052479.Didn't retrieve data . /1386049645.Didn't retrieve data . /1386049638.Didn't retrieve data . /1386049619.Didn't retrieve data . /1386049588.Didn't retrieve data . /1386049569.Didn't retrieve data . /1386049568.Didn't retrieve data . /1386048608.Didn't retrieve data . /1386048577.Didn't retrieve data . /1386048569.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052877', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052814', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052750', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052651', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052547', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052479', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049645', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049638', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049619', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049588', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049569', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049568', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048608', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048577', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048569', None, None, None, None, None, '3784106') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 45 time used for this insertion : 0.01813530921936035 save_final save missing photos in datou_result : time spend for datou_step_exec : 281.35970640182495 time spend to save output : 0.025669336318969727 total time spend for step 8 : 281.3853757381439 step9:crop_condition Fri Sep 26 10:40:23 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 15 ! batch 1 Loaded 143 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758876034_4185575 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.1022145748138428 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758876041_4185575 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.9746789932250977 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758876047_4185575 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.1972029209136963 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758876055_4185575 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.6395094394683838 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 23 About to insert : list_path_to_insert length 23 new photo from crops ! About to upload 23 photos upload in portfolio : 4869462 init cache_photo without model_param we have 23 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758876076_4185575 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 23 photos in the portfolio 4869462 time of upload the photos Elapsed time : 6.982114791870117 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1386052877, 1386052814, 1386052750, 1386052651, 1386052547, 1386052479, 1386049645, 1386049638, 1386049619, 1386049588, 1386049569, 1386049568, 1386048608, 1386048577, 1386048569] Looping around the photos to save general results len do output : 34 /1386323020Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323022Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323023Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323024Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323105Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323124Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323125Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323126Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323127Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323128Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323129Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323213Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323214Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323215Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323216Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323217Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323218Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323219Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323220Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323221Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323222Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323223Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323225Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323226Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323227Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323229Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323230Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323231Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323232Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323233Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323234Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323235Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323236Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386323237Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052877', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052814', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052750', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052651', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052547', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052479', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049645', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049638', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049619', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049588', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049569', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049568', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048608', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048577', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048569', None, None, None, None, None, '3784106') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 117 time used for this insertion : 0.04588747024536133 save_final save missing photos in datou_result : time spend for datou_step_exec : 61.35309982299805 time spend to save output : 0.04765057563781738 total time spend for step 9 : 61.400750398635864 step10:ventilate_hashtags_in_portfolio Fri Sep 26 10:41:25 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 27267728 get user id for portfolio 27267728 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27267728 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','Carton_brun','autre_refus','mal_croppe','plastique','environnement','Teint_Dans_La_Masse','kraft','flou','Carton_gris','metal','cartonnette')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27267728 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','Carton_brun','autre_refus','mal_croppe','plastique','environnement','Teint_Dans_La_Masse','kraft','flou','Carton_gris','metal','cartonnette')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27267728 AND mptpi.`type`=4417 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','Carton_brun','autre_refus','mal_croppe','plastique','environnement','Teint_Dans_La_Masse','kraft','flou','Carton_gris','metal','cartonnette')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27267728 AND mptpi.`type`=4417 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','Carton_brun','autre_refus','mal_croppe','plastique','environnement','Teint_Dans_La_Masse','kraft','flou','Carton_gris','metal','cartonnette')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://marlene.fotonower.com/velours/27270837,27270838,27270839,27270840,27270841,27270842,27270843,27270844,27270845,27270846,27270847,27270848?tags=papier,Carton_brun,autre_refus,mal_croppe,plastique,environnement,Teint_Dans_La_Masse,kraft,flou,Carton_gris,metal,cartonnette Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1386052877, 1386052814, 1386052750, 1386052651, 1386052547, 1386052479, 1386049645, 1386049638, 1386049619, 1386049588, 1386049569, 1386049568, 1386048608, 1386048577, 1386048569] Looping around the photos to save general results len do output : 1 /27267728. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052877', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052814', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052750', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052651', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052547', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052479', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049645', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049638', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049619', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049588', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049569', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049568', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048608', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048577', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048569', None, None, None, None, None, '3784106') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 16 time used for this insertion : 0.0171353816986084 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.3916678428649902 time spend to save output : 0.01747727394104004 total time spend for step 10 : 2.4091451168060303 step11:final Fri Sep 26 10:41:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1386052877: ('0.16152385581089376',), 1386052814: ('0.16152385581089376',), 1386052750: ('0.16152385581089376',), 1386052651: ('0.16152385581089376',), 1386052547: ('0.16152385581089376',), 1386052479: ('0.16152385581089376',), 1386049645: ('0.16152385581089376',), 1386049638: ('0.16152385581089376',), 1386049619: ('0.16152385581089376',), 1386049588: ('0.16152385581089376',), 1386049569: ('0.16152385581089376',), 1386049568: ('0.16152385581089376',), 1386048608: ('0.16152385581089376',), 1386048577: ('0.16152385581089376',), 1386048569: ('0.16152385581089376',)} new output for save of step final : {1386052877: ('0.16152385581089376',), 1386052814: ('0.16152385581089376',), 1386052750: ('0.16152385581089376',), 1386052651: ('0.16152385581089376',), 1386052547: ('0.16152385581089376',), 1386052479: ('0.16152385581089376',), 1386049645: ('0.16152385581089376',), 1386049638: ('0.16152385581089376',), 1386049619: ('0.16152385581089376',), 1386049588: ('0.16152385581089376',), 1386049569: ('0.16152385581089376',), 1386049568: ('0.16152385581089376',), 1386048608: ('0.16152385581089376',), 1386048577: ('0.16152385581089376',), 1386048569: ('0.16152385581089376',)} [1386052877, 1386052814, 1386052750, 1386052651, 1386052547, 1386052479, 1386049645, 1386049638, 1386049619, 1386049588, 1386049569, 1386049568, 1386048608, 1386048577, 1386048569] Looping around the photos to save general results len do output : 15 /1386052877.Didn't retrieve data . /1386052814.Didn't retrieve data . /1386052750.Didn't retrieve data . /1386052651.Didn't retrieve data . /1386052547.Didn't retrieve data . /1386052479.Didn't retrieve data . /1386049645.Didn't retrieve data . /1386049638.Didn't retrieve data . /1386049619.Didn't retrieve data . /1386049588.Didn't retrieve data . /1386049569.Didn't retrieve data . /1386049568.Didn't retrieve data . /1386048608.Didn't retrieve data . /1386048577.Didn't retrieve data . /1386048569.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052877', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052814', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052750', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052651', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052547', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052479', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049645', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049638', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049619', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049588', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049569', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049568', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048608', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048577', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048569', None, None, None, None, None, '3784106') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 45 time used for this insertion : 0.018366098403930664 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.13473033905029297 time spend to save output : 0.019728422164916992 total time spend for step 11 : 0.15445876121520996 step12:velours_tree Fri Sep 26 10:41:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.2653639316558838 time spend to save output : 4.9114227294921875e-05 total time spend for step 12 : 0.2654130458831787 step13:send_mail_cod Fri Sep 26 10:41:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin in order to get the selector url, please entre the license of selector results_Auto_P27267728_26-09-2025_10_41_28.pdf 27270826 change filename to text .change filename to text .change filename to text .imagette272708261758876088 27270827 imagette272708271758876088 27270828 change filename to text .change filename to text .imagette272708281758876088 27270831 change filename to text .imagette272708311758876092 27270832 imagette272708321758876092 27270834 change filename to text .imagette272708341758876092 27270835 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272708351758876093 27270825 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272708251758876094 27270830 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette272708301758876096 27270833 change filename to text .change filename to text .imagette272708331758876099 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=27267728 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://marlene.fotonower.com/velours/27270837,27270838,27270839,27270840,27270841,27270842,27270843,27270844,27270845,27270846,27270847,27270848?tags=papier,Carton_brun,autre_refus,mal_croppe,plastique,environnement,Teint_Dans_La_Masse,kraft,flou,Carton_gris,metal,cartonnette your option no_mail is active, we will not send the real mail to your client args[1386052877] : ((1386052877, -0.22591550915341735, 496442774), (1386052877, 0.042370264104989584, 492688767), '0.16152385581089376') We are sending mail with results at report@fotonower.com args[1386052814] : ((1386052814, 0.2078504194651717, 2107752395), (1386052814, 0.004459216212001633, 492688767), '0.16152385581089376') We are sending mail with results at report@fotonower.com args[1386052750] : ((1386052750, 0.11907526003425398, 2107752395), (1386052750, 0.17099418973702882, 492688767), '0.16152385581089376') We are sending mail with results at report@fotonower.com args[1386052651] : ((1386052651, 0.02587901024711609, 2107752395), (1386052651, 0.45489254576316085, 492688767), '0.16152385581089376') We are sending mail with results at report@fotonower.com args[1386052547] : ((1386052547, -0.04478122394578605, 2107752395), (1386052547, -1.6497149410629899, 492688767), '0.16152385581089376') We are sending mail with results at report@fotonower.com args[1386052479] : ((1386052479, -0.07817896327819138, 496442774), (1386052479, 1.0257004945970205, 492688767), '0.16152385581089376') We are sending mail with results at report@fotonower.com args[1386049645] : ((1386049645, -0.22591550915341735, 496442774), (1386049645, 0.042370264104989584, 492688767), '0.16152385581089376') We are sending mail with results at report@fotonower.com args[1386049638] : ((1386049638, 0.2078504194651717, 2107752395), (1386049638, 0.004459216212001633, 492688767), '0.16152385581089376') We are sending mail with results at report@fotonower.com args[1386049619] : ((1386049619, 0.11907526003425398, 2107752395), (1386049619, 0.17099418973702882, 492688767), '0.16152385581089376') We are sending mail with results at report@fotonower.com args[1386049588] : ((1386049588, 0.02587901024711609, 2107752395), (1386049588, 0.45489254576316085, 492688767), '0.16152385581089376') We are sending mail with results at report@fotonower.com args[1386049569] : ((1386049569, -0.04478122394578605, 2107752395), (1386049569, -1.6497149410629899, 492688767), '0.16152385581089376') We are sending mail with results at report@fotonower.com args[1386049568] : ((1386049568, -0.07817896327819138, 496442774), (1386049568, 1.0257004945970205, 492688767), '0.16152385581089376') We are sending mail with results at report@fotonower.com args[1386048608] : ((1386048608, 0.01879351562582684, 2107752395), (1386048608, 0.43780932820796015, 492688767), '0.16152385581089376') We are sending mail with results at report@fotonower.com args[1386048577] : ((1386048577, -0.07673261312681666, 496442774), (1386048577, 0.6391297616167821, 492688767), '0.16152385581089376') We are sending mail with results at report@fotonower.com args[1386048569] : ((1386048569, 0.060462429127345264, 2107752395), (1386048569, -0.37773187146262144, 492688767), '0.16152385581089376') We are sending mail with results at report@fotonower.com refus_total : 0.16152385581089376 2022-04-13 10:29:59 0 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27267728_26-09-2025_10_41_28.pdf results_Auto_P27267728_26-09-2025_10_41_28.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27267728_26-09-2025_10_41_28.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4433','27267728','results_Auto_P27267728_26-09-2025_10_41_28.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27267728_26-09-2025_10_41_28.pdf','pdf','','1.42','0.16152385581089376') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1386052877, 1386052814, 1386052750, 1386052651, 1386052547, 1386052479, 1386049645, 1386049638, 1386049619, 1386049588, 1386049569, 1386049568, 1386048608, 1386048577, 1386048569] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052877', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052814', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052750', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052651', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052547', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052479', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049645', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049638', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049619', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049588', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049569', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049568', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048608', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048577', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048569', None, None, None, None, None, '3784106') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 15 time used for this insertion : 0.016299962997436523 save_final save missing photos in datou_result : time spend for datou_step_exec : 18.119771480560303 time spend to save output : 0.0271756649017334 total time spend for step 13 : 18.146947145462036 step14:split_time_score Fri Sep 26 10:41:46 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('14', 68),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 19092025 27267728 Nombre de photos uploadées : 68 / 23040 (0%) 19092025 27267728 Nombre de photos taguées (types de déchets): 0 / 68 (0%) 19092025 27267728 Nombre de photos taguées (volume) : 0 / 68 (0%) elapsed_time : load_data_split_time_score 1.430511474609375e-06 elapsed_time : order_list_meta_photo_and_scores 5.7220458984375e-06 ???????????????????????????????????????????????????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.003170013427734375 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.22520160675048828 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.16001606594613368 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27066907_20-09-2025_10_10_49.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27066907 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27066907 AND mptpi.`type`=3726 To do Qualite : 0.16152385581089376 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27267728_26-09-2025_10_41_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27267728 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27267728 AND mptpi.`type`=3726 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'19092025': {'nb_upload': 68, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1386052877, 1386052814, 1386052750, 1386052651, 1386052547, 1386052479, 1386049645, 1386049638, 1386049619, 1386049588, 1386049569, 1386049568, 1386048608, 1386048577, 1386048569] Looping around the photos to save general results len do output : 1 /27267728Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052877', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052814', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052750', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052651', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052547', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386052479', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049645', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049638', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049619', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049588', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049569', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386049568', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048608', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048577', None, None, None, None, None, '3784106') ('4433', None, None, None, None, None, None, None, '3784106') ('4433', '27267728', '1386048569', None, None, None, None, None, '3784106') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 16 time used for this insertion : 0.01566457748413086 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.5714945793151855 time spend to save output : 0.015892505645751953 total time spend for step 14 : 0.5873870849609375 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 385.09user 293.98system 12:54.82elapsed 87%CPU (0avgtext+0avgdata 5315556maxresident)k 9999344inputs+422736outputs (273058major+34952392minor)pagefaults 0swaps