python /home/admin/mtr/script_for_cron.py -j default -m 20 -a 'python3 ~/workarea/git/Velours/python/prod/datou.py -j batch_current -a 4433' -s traitement_sts -M 0 -S 0 -U 100,80,95 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/home/admin/workarea/git/apy', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 78618 load datou : 4433 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4433, datou_cur_ids : ['3772636'] with mtr_portfolio_ids : ['27193432'] and first list_photo_ids : [] new path : /proc/78618/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, brightness, blur_detection, crop_condition, thcl, tfhub_classification2, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 15 list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 15 ; length of list_pids : 15 ; length of list_args : 15 time to download the photos : 4.517667770385742 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 14 step1:mask_detect Fri Sep 26 11:36:01 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3021 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-09-26 11:36:05.863323: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-26 11:36:05.896589: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-26 11:36:05.899406: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fa138000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-26 11:36:05.899475: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-26 11:36:05.905225: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-26 11:36:06.105022: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1dde2af0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-26 11:36:06.105084: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-26 11:36:06.106411: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:36:06.108029: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:36:06.139717: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:36:06.159034: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:36:06.163379: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:36:06.197650: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:36:06.202430: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:36:06.260834: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:36:06.262120: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:36:06.262556: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:36:06.263932: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-26 11:36:06.263952: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-26 11:36:06.263963: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-26 11:36:06.265445: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2569 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-09-26 11:36:06.769822: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:36:06.769926: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:36:06.769946: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:36:06.769965: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:36:06.769981: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:36:06.769998: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:36:06.770014: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:36:06.770031: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:36:06.770858: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:36:06.771786: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:36:06.771825: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:36:06.771840: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:36:06.771854: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:36:06.771868: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:36:06.771882: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:36:06.771895: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:36:06.771909: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:36:06.772772: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:36:06.772827: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-26 11:36:06.772836: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-26 11:36:06.772844: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-26 11:36:06.773789: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2569 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-09-26 11:36:23.819969: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:36:25.155620: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:36:27.658687: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:36:27.737191: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:36:27.744011: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:36:27.744044: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:36:27.796950: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:36:27.797023: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:36:27.839479: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.09GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:36:27.839553: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.09GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:36:27.891453: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.15GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:36:27.891509: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.15GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-26 11:36:27.894191: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.51G (1621688320 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:27.894730: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.36G (1459519488 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.229471: W tensorflow/core/common_runtime/bfc_allocator.cc:311] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature. 2025-09-26 11:36:28.276946: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.277632: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.278652: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.279207: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.289915: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.290494: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.291088: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.291647: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.292206: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.292779: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.307790: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.308379: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.370889: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.371487: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.379559: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.380134: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.408843: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.409590: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.410356: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.411180: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.415243: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.415781: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.416312: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.416882: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.419211: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.419231: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-09-26 11:36:28.428045: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.428621: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.437611: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.438185: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.438726: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.439277: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.439838: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:36:28.440482: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.00G (2147483648 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 15 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 17 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 15 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 16 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 7.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 16 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (3040, 4056, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 4056.00000 nb d'objets trouves : 11 Detection mask done ! Trying to reset tf kernel 78811 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1828 tf kernel not reseted sub process len(results) : 15 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 15 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 3021 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.08493399620056152 nb_pixel_total : 299524 time to create 1 rle with new method : 0.22403216361999512 length of segment : 653 time for calcul the mask position with numpy : 0.0008141994476318359 nb_pixel_total : 33623 time to create 1 rle with old method : 0.03966355323791504 length of segment : 180 time for calcul the mask position with numpy : 0.0049648284912109375 nb_pixel_total : 269202 time to create 1 rle with new method : 0.01610088348388672 length of segment : 731 time for calcul the mask position with numpy : 0.004057884216308594 nb_pixel_total : 201853 time to create 1 rle with new method : 0.011420965194702148 length of segment : 541 time for calcul the mask position with numpy : 0.002532958984375 nb_pixel_total : 107646 time to create 1 rle with old method : 0.1378026008605957 length of segment : 374 time for calcul the mask position with numpy : 0.0006539821624755859 nb_pixel_total : 25631 time to create 1 rle with old method : 0.03014969825744629 length of segment : 185 time for calcul the mask position with numpy : 0.007582187652587891 nb_pixel_total : 299914 time to create 1 rle with new method : 0.049658775329589844 length of segment : 679 time for calcul the mask position with numpy : 0.003679990768432617 nb_pixel_total : 180578 time to create 1 rle with new method : 0.010852336883544922 length of segment : 496 time for calcul the mask position with numpy : 0.0025627613067626953 nb_pixel_total : 92167 time to create 1 rle with old method : 0.11385297775268555 length of segment : 430 time for calcul the mask position with numpy : 0.0006699562072753906 nb_pixel_total : 11517 time to create 1 rle with old method : 0.014378547668457031 length of segment : 118 time for calcul the mask position with numpy : 0.001009225845336914 nb_pixel_total : 46864 time to create 1 rle with old method : 0.05455470085144043 length of segment : 236 time for calcul the mask position with numpy : 0.0014598369598388672 nb_pixel_total : 68892 time to create 1 rle with old method : 0.08304810523986816 length of segment : 237 time for calcul the mask position with numpy : 0.00039076805114746094 nb_pixel_total : 8632 time to create 1 rle with old method : 0.014652490615844727 length of segment : 159 time for calcul the mask position with numpy : 0.0014379024505615234 nb_pixel_total : 89935 time to create 1 rle with old method : 0.10642766952514648 length of segment : 484 time for calcul the mask position with numpy : 0.0008294582366943359 nb_pixel_total : 36185 time to create 1 rle with old method : 0.046802520751953125 length of segment : 329 time for calcul the mask position with numpy : 0.09054803848266602 nb_pixel_total : 1104445 time to create 1 rle with new method : 0.3484795093536377 length of segment : 2553 time for calcul the mask position with numpy : 0.0024576187133789062 nb_pixel_total : 97532 time to create 1 rle with old method : 0.10778188705444336 length of segment : 465 time for calcul the mask position with numpy : 0.005806922912597656 nb_pixel_total : 232855 time to create 1 rle with new method : 0.011723041534423828 length of segment : 700 time for calcul the mask position with numpy : 0.0005562305450439453 nb_pixel_total : 14815 time to create 1 rle with old method : 0.018253803253173828 length of segment : 139 time for calcul the mask position with numpy : 0.0004181861877441406 nb_pixel_total : 18772 time to create 1 rle with old method : 0.020794153213500977 length of segment : 125 time for calcul the mask position with numpy : 0.0002448558807373047 nb_pixel_total : 2292 time to create 1 rle with old method : 0.002621173858642578 length of segment : 68 time for calcul the mask position with numpy : 0.003155231475830078 nb_pixel_total : 158646 time to create 1 rle with new method : 0.004420757293701172 length of segment : 442 time for calcul the mask position with numpy : 0.0005698204040527344 nb_pixel_total : 8806 time to create 1 rle with old method : 0.010631322860717773 length of segment : 89 time for calcul the mask position with numpy : 0.0003783702850341797 nb_pixel_total : 8910 time to create 1 rle with old method : 0.010086297988891602 length of segment : 131 time for calcul the mask position with numpy : 0.0012826919555664062 nb_pixel_total : 34723 time to create 1 rle with old method : 0.03846096992492676 length of segment : 218 time for calcul the mask position with numpy : 0.0008728504180908203 nb_pixel_total : 39230 time to create 1 rle with old method : 0.041640281677246094 length of segment : 218 time for calcul the mask position with numpy : 0.007383108139038086 nb_pixel_total : 259590 time to create 1 rle with new method : 0.015240669250488281 length of segment : 586 time for calcul the mask position with numpy : 0.005598783493041992 nb_pixel_total : 136433 time to create 1 rle with old method : 0.17113614082336426 length of segment : 624 time for calcul the mask position with numpy : 0.0004951953887939453 nb_pixel_total : 8908 time to create 1 rle with old method : 0.010385751724243164 length of segment : 172 time for calcul the mask position with numpy : 0.006486654281616211 nb_pixel_total : 216216 time to create 1 rle with new method : 0.016228199005126953 length of segment : 601 time for calcul the mask position with numpy : 0.0030939579010009766 nb_pixel_total : 172905 time to create 1 rle with new method : 0.005446910858154297 length of segment : 429 time for calcul the mask position with numpy : 0.0006046295166015625 nb_pixel_total : 18079 time to create 1 rle with old method : 0.02156829833984375 length of segment : 126 time for calcul the mask position with numpy : 0.003421783447265625 nb_pixel_total : 157031 time to create 1 rle with new method : 0.010802268981933594 length of segment : 509 time for calcul the mask position with numpy : 0.0008623600006103516 nb_pixel_total : 17185 time to create 1 rle with old method : 0.02315807342529297 length of segment : 206 time for calcul the mask position with numpy : 0.0007102489471435547 nb_pixel_total : 31313 time to create 1 rle with old method : 0.03660273551940918 length of segment : 120 time for calcul the mask position with numpy : 0.013540029525756836 nb_pixel_total : 683086 time to create 1 rle with new method : 0.13045239448547363 length of segment : 712 time for calcul the mask position with numpy : 0.0029191970825195312 nb_pixel_total : 119904 time to create 1 rle with old method : 0.13442587852478027 length of segment : 408 time for calcul the mask position with numpy : 0.0005629062652587891 nb_pixel_total : 16725 time to create 1 rle with old method : 0.018800973892211914 length of segment : 152 time for calcul the mask position with numpy : 0.0007827281951904297 nb_pixel_total : 12681 time to create 1 rle with old method : 0.018335819244384766 length of segment : 277 time for calcul the mask position with numpy : 0.020228862762451172 nb_pixel_total : 353138 time to create 1 rle with new method : 0.3025937080383301 length of segment : 2513 time for calcul the mask position with numpy : 0.0009458065032958984 nb_pixel_total : 15420 time to create 1 rle with old method : 0.018702030181884766 length of segment : 224 time for calcul the mask position with numpy : 0.002855062484741211 nb_pixel_total : 146966 time to create 1 rle with old method : 0.176102876663208 length of segment : 302 time for calcul the mask position with numpy : 0.008496522903442383 nb_pixel_total : 373899 time to create 1 rle with new method : 0.011385679244995117 length of segment : 897 time for calcul the mask position with numpy : 0.003568410873413086 nb_pixel_total : 158327 time to create 1 rle with new method : 0.005085468292236328 length of segment : 538 time for calcul the mask position with numpy : 0.0019974708557128906 nb_pixel_total : 55805 time to create 1 rle with old method : 0.0686647891998291 length of segment : 264 time for calcul the mask position with numpy : 0.0018804073333740234 nb_pixel_total : 51503 time to create 1 rle with old method : 0.0609893798828125 length of segment : 349 time for calcul the mask position with numpy : 0.003245830535888672 nb_pixel_total : 134429 time to create 1 rle with old method : 0.1590726375579834 length of segment : 570 time for calcul the mask position with numpy : 0.00513148307800293 nb_pixel_total : 174750 time to create 1 rle with new method : 0.007871150970458984 length of segment : 1019 time for calcul the mask position with numpy : 0.0008587837219238281 nb_pixel_total : 20968 time to create 1 rle with old method : 0.024680614471435547 length of segment : 188 time for calcul the mask position with numpy : 0.0014336109161376953 nb_pixel_total : 51587 time to create 1 rle with old method : 0.05955171585083008 length of segment : 229 time for calcul the mask position with numpy : 0.0023813247680664062 nb_pixel_total : 87934 time to create 1 rle with old method : 0.09957695007324219 length of segment : 358 time for calcul the mask position with numpy : 0.0030732154846191406 nb_pixel_total : 164629 time to create 1 rle with new method : 0.006098508834838867 length of segment : 309 time for calcul the mask position with numpy : 0.0016601085662841797 nb_pixel_total : 53742 time to create 1 rle with old method : 0.06282329559326172 length of segment : 320 time for calcul the mask position with numpy : 0.0007593631744384766 nb_pixel_total : 23019 time to create 1 rle with old method : 0.02678370475769043 length of segment : 216 time for calcul the mask position with numpy : 0.0042765140533447266 nb_pixel_total : 187289 time to create 1 rle with new method : 0.006880998611450195 length of segment : 472 time for calcul the mask position with numpy : 0.0002639293670654297 nb_pixel_total : 7643 time to create 1 rle with old method : 0.008877992630004883 length of segment : 121 time for calcul the mask position with numpy : 0.0005557537078857422 nb_pixel_total : 15911 time to create 1 rle with old method : 0.0187530517578125 length of segment : 87 time for calcul the mask position with numpy : 0.0008707046508789062 nb_pixel_total : 22806 time to create 1 rle with old method : 0.026383399963378906 length of segment : 260 time for calcul the mask position with numpy : 0.03943777084350586 nb_pixel_total : 603769 time to create 1 rle with new method : 0.43779420852661133 length of segment : 2115 time for calcul the mask position with numpy : 0.004689455032348633 nb_pixel_total : 252944 time to create 1 rle with new method : 0.00644373893737793 length of segment : 468 time for calcul the mask position with numpy : 0.0005745887756347656 nb_pixel_total : 31185 time to create 1 rle with old method : 0.036112308502197266 length of segment : 208 time for calcul the mask position with numpy : 0.0011382102966308594 nb_pixel_total : 53032 time to create 1 rle with old method : 0.06346487998962402 length of segment : 240 time for calcul the mask position with numpy : 0.0011749267578125 nb_pixel_total : 39161 time to create 1 rle with old method : 0.04837799072265625 length of segment : 419 time for calcul the mask position with numpy : 0.016464948654174805 nb_pixel_total : 779040 time to create 1 rle with new method : 0.04951763153076172 length of segment : 1239 time for calcul the mask position with numpy : 0.006730079650878906 nb_pixel_total : 276281 time to create 1 rle with new method : 0.015754222869873047 length of segment : 1171 time for calcul the mask position with numpy : 0.00038123130798339844 nb_pixel_total : 10481 time to create 1 rle with old method : 0.014616966247558594 length of segment : 90 time for calcul the mask position with numpy : 0.0060577392578125 nb_pixel_total : 253350 time to create 1 rle with new method : 0.01166081428527832 length of segment : 494 time for calcul the mask position with numpy : 0.005549192428588867 nb_pixel_total : 252122 time to create 1 rle with new method : 0.011554241180419922 length of segment : 624 time for calcul the mask position with numpy : 0.025583505630493164 nb_pixel_total : 1328090 time to create 1 rle with new method : 0.24890661239624023 length of segment : 2328 time for calcul the mask position with numpy : 0.004891872406005859 nb_pixel_total : 250708 time to create 1 rle with new method : 0.007890462875366211 length of segment : 477 time for calcul the mask position with numpy : 0.007645368576049805 nb_pixel_total : 267814 time to create 1 rle with new method : 0.017185449600219727 length of segment : 585 time for calcul the mask position with numpy : 0.0023665428161621094 nb_pixel_total : 123242 time to create 1 rle with old method : 0.13890290260314941 length of segment : 393 time for calcul the mask position with numpy : 0.0032188892364501953 nb_pixel_total : 126081 time to create 1 rle with old method : 0.1463181972503662 length of segment : 435 time for calcul the mask position with numpy : 0.0007569789886474609 nb_pixel_total : 28135 time to create 1 rle with old method : 0.03200173377990723 length of segment : 244 time for calcul the mask position with numpy : 0.0003066062927246094 nb_pixel_total : 2751 time to create 1 rle with old method : 0.0034766197204589844 length of segment : 101 time for calcul the mask position with numpy : 0.0006084442138671875 nb_pixel_total : 17951 time to create 1 rle with old method : 0.02165508270263672 length of segment : 175 time for calcul the mask position with numpy : 0.008814096450805664 nb_pixel_total : 265142 time to create 1 rle with new method : 0.026019811630249023 length of segment : 1468 time for calcul the mask position with numpy : 0.009664773941040039 nb_pixel_total : 372990 time to create 1 rle with new method : 0.03743267059326172 length of segment : 609 time for calcul the mask position with numpy : 0.009306192398071289 nb_pixel_total : 390501 time to create 1 rle with new method : 0.013855457305908203 length of segment : 731 time for calcul the mask position with numpy : 0.0003390312194824219 nb_pixel_total : 6796 time to create 1 rle with old method : 0.008348941802978516 length of segment : 84 time for calcul the mask position with numpy : 0.0055027008056640625 nb_pixel_total : 292900 time to create 1 rle with new method : 0.007986068725585938 length of segment : 497 time for calcul the mask position with numpy : 0.0013194084167480469 nb_pixel_total : 40311 time to create 1 rle with old method : 0.04832577705383301 length of segment : 247 time for calcul the mask position with numpy : 0.012064218521118164 nb_pixel_total : 422081 time to create 1 rle with new method : 0.017902374267578125 length of segment : 684 time for calcul the mask position with numpy : 0.0005104541778564453 nb_pixel_total : 14763 time to create 1 rle with old method : 0.017622709274291992 length of segment : 116 time for calcul the mask position with numpy : 0.002270936965942383 nb_pixel_total : 116606 time to create 1 rle with old method : 0.13367557525634766 length of segment : 372 time for calcul the mask position with numpy : 0.03994274139404297 nb_pixel_total : 1405275 time to create 1 rle with new method : 0.9208605289459229 length of segment : 2465 time for calcul the mask position with numpy : 0.0013740062713623047 nb_pixel_total : 41841 time to create 1 rle with old method : 0.04826664924621582 length of segment : 169 time for calcul the mask position with numpy : 0.0031135082244873047 nb_pixel_total : 199006 time to create 1 rle with new method : 0.008775949478149414 length of segment : 492 time for calcul the mask position with numpy : 0.0074846744537353516 nb_pixel_total : 271588 time to create 1 rle with new method : 0.012996196746826172 length of segment : 510 time for calcul the mask position with numpy : 0.0010416507720947266 nb_pixel_total : 39793 time to create 1 rle with old method : 0.047731637954711914 length of segment : 300 time for calcul the mask position with numpy : 0.0013053417205810547 nb_pixel_total : 50905 time to create 1 rle with old method : 0.06372284889221191 length of segment : 326 time for calcul the mask position with numpy : 0.011077404022216797 nb_pixel_total : 377719 time to create 1 rle with new method : 0.019077062606811523 length of segment : 1116 time for calcul the mask position with numpy : 0.00047469139099121094 nb_pixel_total : 27006 time to create 1 rle with old method : 0.0318751335144043 length of segment : 260 time for calcul the mask position with numpy : 0.001068115234375 nb_pixel_total : 61884 time to create 1 rle with old method : 0.07089614868164062 length of segment : 590 time for calcul the mask position with numpy : 0.0037953853607177734 nb_pixel_total : 108889 time to create 1 rle with old method : 0.12691760063171387 length of segment : 585 time for calcul the mask position with numpy : 0.014524221420288086 nb_pixel_total : 480031 time to create 1 rle with new method : 0.01726841926574707 length of segment : 1013 time for calcul the mask position with numpy : 0.00038552284240722656 nb_pixel_total : 10580 time to create 1 rle with old method : 0.012410163879394531 length of segment : 107 time for calcul the mask position with numpy : 0.00055694580078125 nb_pixel_total : 16811 time to create 1 rle with old method : 0.019894123077392578 length of segment : 179 time for calcul the mask position with numpy : 0.0011951923370361328 nb_pixel_total : 41499 time to create 1 rle with old method : 0.06685066223144531 length of segment : 272 time for calcul the mask position with numpy : 0.00434565544128418 nb_pixel_total : 205078 time to create 1 rle with new method : 0.006741046905517578 length of segment : 488 time for calcul the mask position with numpy : 0.014475107192993164 nb_pixel_total : 500157 time to create 1 rle with new method : 0.02800774574279785 length of segment : 944 time for calcul the mask position with numpy : 0.03779911994934082 nb_pixel_total : 976396 time to create 1 rle with new method : 0.2722585201263428 length of segment : 2242 time for calcul the mask position with numpy : 0.006523609161376953 nb_pixel_total : 433157 time to create 1 rle with new method : 0.013733863830566406 length of segment : 983 time for calcul the mask position with numpy : 0.001424551010131836 nb_pixel_total : 49513 time to create 1 rle with old method : 0.05719590187072754 length of segment : 274 time for calcul the mask position with numpy : 0.008228063583374023 nb_pixel_total : 283698 time to create 1 rle with new method : 0.01767277717590332 length of segment : 920 time for calcul the mask position with numpy : 0.008372306823730469 nb_pixel_total : 492814 time to create 1 rle with new method : 0.018929719924926758 length of segment : 960 time for calcul the mask position with numpy : 0.0031876564025878906 nb_pixel_total : 142217 time to create 1 rle with old method : 0.16167235374450684 length of segment : 473 time for calcul the mask position with numpy : 0.002524137496948242 nb_pixel_total : 36444 time to create 1 rle with old method : 0.040557861328125 length of segment : 471 time for calcul the mask position with numpy : 0.0011556148529052734 nb_pixel_total : 65876 time to create 1 rle with old method : 0.07439661026000977 length of segment : 323 time for calcul the mask position with numpy : 0.0006210803985595703 nb_pixel_total : 23924 time to create 1 rle with old method : 0.027255535125732422 length of segment : 206 time for calcul the mask position with numpy : 0.0004336833953857422 nb_pixel_total : 11914 time to create 1 rle with old method : 0.013978719711303711 length of segment : 175 time for calcul the mask position with numpy : 0.0012366771697998047 nb_pixel_total : 38922 time to create 1 rle with old method : 0.04380536079406738 length of segment : 298 time for calcul the mask position with numpy : 0.0013439655303955078 nb_pixel_total : 51418 time to create 1 rle with old method : 0.05926370620727539 length of segment : 238 time for calcul the mask position with numpy : 0.001664876937866211 nb_pixel_total : 60611 time to create 1 rle with old method : 0.06830191612243652 length of segment : 220 time for calcul the mask position with numpy : 0.003155946731567383 nb_pixel_total : 95253 time to create 1 rle with old method : 0.11360836029052734 length of segment : 401 time for calcul the mask position with numpy : 0.037384748458862305 nb_pixel_total : 581837 time to create 1 rle with new method : 0.5335099697113037 length of segment : 2058 time for calcul the mask position with numpy : 0.003782033920288086 nb_pixel_total : 212386 time to create 1 rle with new method : 0.008866548538208008 length of segment : 487 time for calcul the mask position with numpy : 0.0005412101745605469 nb_pixel_total : 14479 time to create 1 rle with old method : 0.01729416847229004 length of segment : 123 time for calcul the mask position with numpy : 0.002450227737426758 nb_pixel_total : 82053 time to create 1 rle with old method : 0.09484243392944336 length of segment : 417 time for calcul the mask position with numpy : 0.003559112548828125 nb_pixel_total : 153039 time to create 1 rle with new method : 0.00840139389038086 length of segment : 941 time for calcul the mask position with numpy : 0.0007054805755615234 nb_pixel_total : 17918 time to create 1 rle with old method : 0.021275043487548828 length of segment : 134 time for calcul the mask position with numpy : 0.003161907196044922 nb_pixel_total : 81668 time to create 1 rle with old method : 0.09794974327087402 length of segment : 383 time for calcul the mask position with numpy : 0.0006048679351806641 nb_pixel_total : 19657 time to create 1 rle with old method : 0.022724390029907227 length of segment : 207 time for calcul the mask position with numpy : 0.009301424026489258 nb_pixel_total : 482118 time to create 1 rle with new method : 0.02741551399230957 length of segment : 1210 time for calcul the mask position with numpy : 0.0023326873779296875 nb_pixel_total : 102939 time to create 1 rle with old method : 0.11528730392456055 length of segment : 364 time for calcul the mask position with numpy : 0.009316205978393555 nb_pixel_total : 450964 time to create 1 rle with new method : 0.01672220230102539 length of segment : 795 time for calcul the mask position with numpy : 0.003996133804321289 nb_pixel_total : 222876 time to create 1 rle with new method : 0.0067713260650634766 length of segment : 456 time for calcul the mask position with numpy : 0.0012278556823730469 nb_pixel_total : 43850 time to create 1 rle with old method : 0.05087566375732422 length of segment : 357 time for calcul the mask position with numpy : 0.0014104843139648438 nb_pixel_total : 80183 time to create 1 rle with old method : 0.0955038070678711 length of segment : 236 time for calcul the mask position with numpy : 0.0008902549743652344 nb_pixel_total : 35341 time to create 1 rle with old method : 0.042351484298706055 length of segment : 144 time for calcul the mask position with numpy : 0.003663778305053711 nb_pixel_total : 197492 time to create 1 rle with new method : 0.0075876712799072266 length of segment : 485 time for calcul the mask position with numpy : 0.0018470287322998047 nb_pixel_total : 75418 time to create 1 rle with old method : 0.08658432960510254 length of segment : 410 time for calcul the mask position with numpy : 0.004358530044555664 nb_pixel_total : 124949 time to create 1 rle with old method : 0.161118745803833 length of segment : 566 time for calcul the mask position with numpy : 0.0008099079132080078 nb_pixel_total : 18356 time to create 1 rle with old method : 0.021188974380493164 length of segment : 204 time for calcul the mask position with numpy : 0.0008797645568847656 nb_pixel_total : 26081 time to create 1 rle with old method : 0.030829906463623047 length of segment : 239 time for calcul the mask position with numpy : 0.0007500648498535156 nb_pixel_total : 30600 time to create 1 rle with old method : 0.0358431339263916 length of segment : 216 time for calcul the mask position with numpy : 0.0005006790161132812 nb_pixel_total : 19016 time to create 1 rle with old method : 0.02195906639099121 length of segment : 263 time for calcul the mask position with numpy : 0.03554201126098633 nb_pixel_total : 1583388 time to create 1 rle with new method : 0.17457294464111328 length of segment : 1838 time for calcul the mask position with numpy : 0.0006787776947021484 nb_pixel_total : 18167 time to create 1 rle with old method : 0.020559072494506836 length of segment : 215 time for calcul the mask position with numpy : 0.002804279327392578 nb_pixel_total : 144163 time to create 1 rle with old method : 0.16413593292236328 length of segment : 433 time for calcul the mask position with numpy : 0.002763509750366211 nb_pixel_total : 134857 time to create 1 rle with old method : 0.15424394607543945 length of segment : 373 time for calcul the mask position with numpy : 0.0014886856079101562 nb_pixel_total : 32516 time to create 1 rle with old method : 0.03798055648803711 length of segment : 309 time spent for convertir_results : 24.98852014541626 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 142 chid ids of type : 3663 Number RLEs to save : 73615 save missing photos in datou_result : time spend for datou_step_exec : 123.40902996063232 time spend to save output : 4.557893753051758 total time spend for step 1 : 127.96692371368408 step2:brightness Fri Sep 26 11:38:09 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1758879357_78618_1385898658_34967bb2848c428aa1b994247aae1106.jpg treat image : temp/1758879357_78618_1385898622_1023cde7ae50ee960c667bcec5aa6ec9.jpg treat image : temp/1758879357_78618_1385898582_c2510fe01264554b5fd6e30e8ecaa308.jpg treat image : temp/1758879357_78618_1385898539_44ad7847b5702cecc47632d48a470904.jpg treat image : temp/1758879357_78618_1385898498_f4521b4954847966c7985e93f463bb93.jpg treat image : temp/1758879357_78618_1385898459_e07cb72bed42d9efc6e345e9382cbed3.jpg treat image : temp/1758879357_78618_1385897627_8dba210e90e76977be469630bbdef0ac.jpg treat image : temp/1758879357_78618_1385897622_01a5ac29d7ea800d97d44bbfd975f123.jpg treat image : temp/1758879357_78618_1385897613_1d9b15bbaafedb9fbbb2059433621b79.jpg treat image : temp/1758879357_78618_1385897612_e4101cc1b76dd9949d64b2ada84ca9f3.jpg treat image : temp/1758879357_78618_1385897611_b2f589a7c34668b8f2e64868773dab18.jpg treat image : temp/1758879357_78618_1385897609_cce1e2f3e6148e21118a43eb581ffd66.jpg treat image : temp/1758879357_78618_1385897405_16d492171aabcc51ddb4f2db100735a2.jpg treat image : temp/1758879357_78618_1385897404_1fcdae9935a48d7bb23d8533ea413cb3.jpg treat image : temp/1758879357_78618_1385897379_77dbaa6f19c2fd69db9b0acb3a874025.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 15 time used for this insertion : 0.012825250625610352 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 15 time used for this insertion : 0.015190362930297852 save missing photos in datou_result : time spend for datou_step_exec : 25.175824403762817 time spend to save output : 0.30344629287719727 total time spend for step 2 : 25.479270696640015 step3:blur_detection Fri Sep 26 11:38:35 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1758879357_78618_1385898658_34967bb2848c428aa1b994247aae1106.jpg resize: (3040, 4056) 1385898658 -0.1849462224109979 treat image : temp/1758879357_78618_1385898622_1023cde7ae50ee960c667bcec5aa6ec9.jpg resize: (3040, 4056) 1385898622 0.07890119766233548 treat image : temp/1758879357_78618_1385898582_c2510fe01264554b5fd6e30e8ecaa308.jpg resize: (3040, 4056) 1385898582 0.3429494918624544 treat image : temp/1758879357_78618_1385898539_44ad7847b5702cecc47632d48a470904.jpg resize: (3040, 4056) 1385898539 0.35238906227014066 treat image : temp/1758879357_78618_1385898498_f4521b4954847966c7985e93f463bb93.jpg resize: (3040, 4056) 1385898498 0.29104777152107375 treat image : temp/1758879357_78618_1385898459_e07cb72bed42d9efc6e345e9382cbed3.jpg resize: (3040, 4056) 1385898459 0.5460009509167261 treat image : temp/1758879357_78618_1385897627_8dba210e90e76977be469630bbdef0ac.jpg resize: (3040, 4056) 1385897627 0.4803432031216644 treat image : temp/1758879357_78618_1385897622_01a5ac29d7ea800d97d44bbfd975f123.jpg resize: (3040, 4056) 1385897622 0.45213623997877644 treat image : temp/1758879357_78618_1385897613_1d9b15bbaafedb9fbbb2059433621b79.jpg resize: (3040, 4056) 1385897613 0.3610692242860735 treat image : temp/1758879357_78618_1385897612_e4101cc1b76dd9949d64b2ada84ca9f3.jpg resize: (3040, 4056) 1385897612 0.33443143424367916 treat image : temp/1758879357_78618_1385897611_b2f589a7c34668b8f2e64868773dab18.jpg resize: (3040, 4056) 1385897611 0.18957091587469432 treat image : temp/1758879357_78618_1385897609_cce1e2f3e6148e21118a43eb581ffd66.jpg resize: (3040, 4056) 1385897609 0.21590676778930154 treat image : temp/1758879357_78618_1385897405_16d492171aabcc51ddb4f2db100735a2.jpg resize: (3040, 4056) 1385897405 0.33879943153998204 treat image : temp/1758879357_78618_1385897404_1fcdae9935a48d7bb23d8533ea413cb3.jpg resize: (3040, 4056) 1385897404 0.39687990354867203 treat image : temp/1758879357_78618_1385897379_77dbaa6f19c2fd69db9b0acb3a874025.jpg resize: (3040, 4056) 1385897379 0.5392607326932135 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 15 time used for this insertion : 0.012015342712402344 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 15 time used for this insertion : 0.011966705322265625 save missing photos in datou_result : time spend for datou_step_exec : 93.6438820362091 time spend to save output : 0.05106973648071289 total time spend for step 3 : 93.69495177268982 step4:crop_condition Fri Sep 26 11:40:08 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 15 ! batch 1 Loaded 142 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 18 About to insert : list_path_to_insert length 18 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 12 About to insert : list_path_to_insert length 12 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 35 About to insert : list_path_to_insert length 35 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 31 About to insert : list_path_to_insert length 31 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1385898658, 1385898622, 1385898582, 1385898539, 1385898498, 1385898459, 1385897627, 1385897622, 1385897613, 1385897612, 1385897611, 1385897609, 1385897405, 1385897404, 1385897379] Looping around the photos to save general results len do output : 109 /-3976659303Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659305Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659307Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659316Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659324Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659328Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659338Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659346Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659366Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659367Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659377Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659383Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659398Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659400Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659412Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659439Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659440Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659441Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659345Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659370Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659372Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659407Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659416Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659420Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659431Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659309Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659339Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659352Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659363Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659375Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659389Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659391Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659396Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659397Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659408Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659409Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659433Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659319Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659320Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659327Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659330Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659333Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659335Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659344Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659347Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659348Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659349Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659351Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659353Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659354Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659357Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659362Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659369Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659373Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659374Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659381Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659385Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659387Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659394Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659395Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659401Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659402Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659414Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659417Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659419Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659421Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659422Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659427Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659429Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659434Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659435Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659442Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659304Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659306Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659308Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659311Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659312Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659313Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659314Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659315Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659322Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659325Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659326Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659336Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659356Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659359Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659360Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659368Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659376Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659378Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659379Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659380Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659384Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659386Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659390Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659392Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659399Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659403Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659411Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659432Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659436Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659437Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659438Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659317Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659329Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659405Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659410Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659428Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3976659443Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4433', None, None, None, None, None, None, None, '3772636') ('4433', '27193432', '1385898658', None, None, None, None, None, '3772636') ('4433', None, None, None, None, None, None, None, '3772636') ('4433', '27193432', '1385898622', None, None, None, None, None, '3772636') ('4433', None, None, None, None, None, None, None, '3772636') ('4433', '27193432', '1385898582', None, None, None, None, None, '3772636') ('4433', None, None, None, None, None, None, None, '3772636') ('4433', '27193432', '1385898539', None, None, None, None, None, '3772636') ('4433', None, None, None, None, None, None, None, '3772636') ('4433', '27193432', '1385898498', None, None, None, None, None, '3772636') ('4433', None, None, None, None, None, None, None, '3772636') ('4433', '27193432', '1385898459', None, None, None, None, None, '3772636') ('4433', None, None, None, None, None, None, None, '3772636') ('4433', '27193432', '1385897627', None, None, None, None, None, '3772636') ('4433', None, None, None, None, None, None, None, '3772636') ('4433', '27193432', '1385897622', None, None, None, None, None, '3772636') ('4433', None, None, None, None, None, None, None, '3772636') ('4433', '27193432', '1385897613', None, None, None, None, None, '3772636') ('4433', None, None, None, None, None, None, None, '3772636') ('4433', '27193432', '1385897612', None, None, None, None, None, '3772636') ('4433', None, None, None, None, None, None, None, '3772636') ('4433', '27193432', '1385897611', None, None, None, None, None, '3772636') ('4433', None, None, None, None, None, None, None, '3772636') ('4433', '27193432', '1385897609', None, None, None, None, None, '3772636') ('4433', None, None, None, None, None, None, None, '3772636') ('4433', '27193432', '1385897405', None, None, None, None, None, '3772636') ('4433', None, None, None, None, None, None, None, '3772636') ('4433', '27193432', '1385897404', None, None, None, None, None, '3772636') ('4433', None, None, None, None, None, None, None, '3772636') ('4433', '27193432', '1385897379', None, None, None, None, None, '3772636') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 342 time used for this insertion : 0.038790225982666016 save_final save missing photos in datou_result : time spend for datou_step_exec : 113.71383595466614 time spend to save output : 0.04406929016113281 total time spend for step 4 : 113.75790524482727 step5:thcl Fri Sep 26 11:42:02 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 1 we are using the classfication for multi_thcl [2456] time to import caffe and check if the image exist : 0.018513202667236328 time to convert the images to numpy array : 0.2561783790588379 time to import caffe and check if the image exist : 0.01978445053100586 time to convert the images to numpy array : 1.0731523036956787 time to import caffe and check if the image exist : 0.014011859893798828 time to convert the images to numpy array : 1.128241777420044 time to import caffe and check if the image exist : 0.018611907958984375 time to convert the images to numpy array : 1.1290233135223389 time to import caffe and check if the image exist : 0.02276468276977539 time to convert the images to numpy array : 1.140268325805664 time to import caffe and check if the image exist : 0.02225804328918457 time to convert the images to numpy array : 1.2056617736816406 time to import caffe and check if the image exist : 0.024091005325317383 time to convert the images to numpy array : 1.2970311641693115 time to import caffe and check if the image exist : 0.022124052047729492 time to convert the images to numpy array : 1.3567841053009033 time to import caffe and check if the image exist : 0.023678302764892578 time to convert the images to numpy array : 1.745072364807129 time to import caffe and check if the image exist : 0.02073383331298828 time to convert the images to numpy array : 1.816868543624878 total time to convert the images to numpy array : 2.290360689163208 list photo_ids error: [] list photo_ids correct : [-3976659313, -3976659314, -3976659315, -3976659322, -3976659325, -3976659326, -3976659336, -3976659356, -3976659359, -3976659360, -3976659368, -3976659422, -3976659427, -3976659429, -3976659434, -3976659435, -3976659442, -3976659304, -3976659306, -3976659308, -3976659311, -3976659312, -3976659376, -3976659378, -3976659379, -3976659380, -3976659384, -3976659386, -3976659390, -3976659392, -3976659399, -3976659403, -3976659411, -3976659432, -3976659436, -3976659437, -3976659438, -3976659317, -3976659329, -3976659405, -3976659410, -3976659428, -3976659443, -3976659416, -3976659420, -3976659431, -3976659309, -3976659339, -3976659352, -3976659363, -3976659375, -3976659389, -3976659391, -3976659396, -3976659347, -3976659348, -3976659349, -3976659351, -3976659353, -3976659354, -3976659357, -3976659362, -3976659369, -3976659373, -3976659374, -3976659397, -3976659408, -3976659409, -3976659433, -3976659319, -3976659320, -3976659327, -3976659330, -3976659333, -3976659335, -3976659344, -3976659381, -3976659385, -3976659387, -3976659394, -3976659395, -3976659401, -3976659402, -3976659414, -3976659417, -3976659419, -3976659421, -3976659303, -3976659305, -3976659307, -3976659316, -3976659324, -3976659328, -3976659338, -3976659346, -3976659366, -3976659367, -3976659377, -3976659383, -3976659398, -3976659400, -3976659412, -3976659439, -3976659440, -3976659441, -3976659345, -3976659370, -3976659372, -3976659407] number of photos to traite : 109 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 5202 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/:/home/admin/workarea/git/apy/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 4983 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 2.489663600921631 time used to do the prediction : 0.2854623794555664 we don't save the descriptors for this thcl 2456 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 3.314018249511719e-05 save missing photos in datou_result : time spend for datou_step_exec : 11.74849796295166 time spend to save output : 0.0004668235778808594 total time spend for step 5 : 11.748964786529541 step6:tfhub_classification2 Fri Sep 26 11:42:14 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou_step TFHub with tf2 ! we are using the classfication for only one thcl 3501 begin to check gpu status inside check gpu memory 2025-09-26 11:42:19.670523: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-26 11:42:19.675738: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:42:19.675911: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:42:19.675977: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:42:19.696234: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:42:19.696587: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:42:19.730561: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:42:19.735295: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:42:19.784831: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:42:19.785996: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:42:19.786854: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-26 11:42:19.820529: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-26 11:42:19.822713: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fa024000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-26 11:42:19.822755: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-26 11:42:19.827248: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x21902750 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-26 11:42:19.827282: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-26 11:42:19.827886: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-26 11:42:19.827967: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:42:19.827988: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-26 11:42:19.828090: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-26 11:42:19.828113: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-26 11:42:19.828142: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-26 11:42:19.828171: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-26 11:42:19.828202: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-26 11:42:19.829026: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-26 11:42:19.829466: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-26 11:42:19.830026: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-26 11:42:19.830044: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-26 11:42:19.830054: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-26 11:42:19.830896: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3096 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) l 3637 free memory gpu now : 4981 max_wait_temp : 1 max_wait : 5 1 Physical GPUs, 1 Logical GPUs tagging for thcl : 3501 To do loadFromThcl(), then load ParamDescType : thcl3501 thcls : [{'id': 3501, 'mtr_user_id': 31, 'name': 'learn_papier_mixte_centre_tf_b4_220223_3', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '9087724,9087727,9087725,9087729,9087726,9087723,9087722,9087720,9087721,9087728', 'photo_hashtag_type': 4539, 'photo_desc_type': 5759, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 3501, 'mtr_user_id': 31, 'name': 'learn_papier_mixte_centre_tf_b4_220223_3', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '9087724,9087727,9087725,9087729,9087726,9087723,9087722,9087720,9087721,9087728', 'photo_hashtag_type': 4539, 'photo_desc_type': 5759, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5759 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5759, 'learn_papier_mixte_centre_tf_b4_220223_3', 1792, 1792, 'learn_papier_mixte_centre_tf_b4_220223_3', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 2, 22, 13, 54, 32), datetime.datetime(2023, 2, 22, 13, 54, 32)) model_name : learn_papier_mixte_centre_tf_b4_220223_3 model_param file didn't exist model_name : learn_papier_mixte_centre_tf_b4_220223_3 model_type : tf_classification2 list file need : ['Confusion_Matrix.png', 'Precision_Recall_Carton_brun.jpg', 'Precision_Recall_Carton_gris.jpg', 'Precision_Recall_Teint_Dans_La_Masse.jpg', 'Precision_Recall_autre_refus.jpg', 'Precision_Recall_cartonnette.jpg', 'Precision_Recall_environnement.jpg', 'Precision_Recall_kraft.jpg', 'Precision_Recall_metal.jpg', 'Precision_Recall_papier.jpg', 'Precision_Recall_plastique.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file exist in s3 : ['Confusion_Matrix.png', 'Precision_Recall_Carton_brun.jpg', 'Precision_Recall_Carton_gris.jpg', 'Precision_Recall_Teint_Dans_La_Masse.jpg', 'Precision_Recall_autre_refus.jpg', 'Precision_Recall_cartonnette.jpg', 'Precision_Recall_environnement.jpg', 'Precision_Recall_kraft.jpg', 'Precision_Recall_metal.jpg', 'Precision_Recall_papier.jpg', 'Precision_Recall_plastique.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file manque in s3 : [] 2025-09-26 11:42:35.094641: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.02G (3246391296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.120280: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.72G (2921752064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.120953: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.45G (2629576704 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.121576: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.20G (2366618880 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.122199: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2129957120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.122867: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.79G (1916961536 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.123508: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.61G (1725265408 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.124150: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.45G (1552738816 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.124813: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.30G (1397465088 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.125450: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.17G (1257718528 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.126085: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.05G (1131946752 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.126705: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 971.56M (1018752256 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.127323: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 874.40M (916877056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.127963: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 786.96M (825189376 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.128611: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 708.27M (742670592 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.129249: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 637.44M (668403712 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.129825: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 573.70M (601563392 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.130315: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 516.33M (541407232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.130825: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 464.69M (487266560 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.131330: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 418.22M (438540032 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.131819: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 376.40M (394686208 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.132311: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 338.76M (355217664 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.132836: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 304.89M (319696128 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.133342: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 274.40M (287726592 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.133868: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 246.96M (258953984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.134437: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 222.26M (233058816 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.134958: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 200.04M (209753088 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.135460: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 180.03M (188777984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.135973: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 162.03M (169900288 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.136495: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 145.83M (152910336 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.136997: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 131.24M (137619456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.137510: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 118.12M (123857664 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.137985: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 106.31M (111472128 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.138436: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 95.68M (100325120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-26 11:42:35.138886: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 86.11M (90292736 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3 /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Confusion_Matrix.png size_local : 121269 size in s3 : 121269 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:17 Confusion_Matrix.png already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_Carton_brun.jpg size_local : 76104 size in s3 : 76104 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_Carton_brun.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_Carton_gris.jpg size_local : 82912 size in s3 : 82912 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_Carton_gris.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_Teint_Dans_La_Masse.jpg size_local : 75423 size in s3 : 75423 create time local : 2023-03-03 12:26:14 create time in s3 : 2023-02-22 12:54:25 Precision_Recall_Teint_Dans_La_Masse.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_autre_refus.jpg size_local : 78883 size in s3 : 78883 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_autre_refus.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_cartonnette.jpg size_local : 81791 size in s3 : 81791 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:30 Precision_Recall_cartonnette.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_environnement.jpg size_local : 59936 size in s3 : 59936 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:17 Precision_Recall_environnement.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_kraft.jpg size_local : 80280 size in s3 : 80280 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:25 Precision_Recall_kraft.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_metal.jpg size_local : 77736 size in s3 : 77736 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_metal.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_papier.jpg size_local : 69041 size in s3 : 69041 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Precision_Recall_papier.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Precision_Recall_plastique.jpg size_local : 70770 size in s3 : 70770 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:17 Precision_Recall_plastique.jpg already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/Result_Summary.txt size_local : 1092 size in s3 : 1092 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 Result_Summary.txt already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/checkpoint size_local : 99 size in s3 : 99 create time local : 2023-03-03 12:26:15 create time in s3 : 2023-02-22 12:54:31 checkpoint already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_checkpoint.ckpt.data-00000-of-00002 size_local : 429801 size in s3 : 429801 create time local : 2023-03-03 12:26:16 create time in s3 : 2023-02-22 12:54:30 model_checkpoint.ckpt.data-00000-of-00002 already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_checkpoint.ckpt.data-00001-of-00002 size_local : 141033188 size in s3 : 141033188 create time local : 2023-03-03 12:26:17 create time in s3 : 2023-02-22 12:54:25 model_checkpoint.ckpt.data-00001-of-00002 already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_checkpoint.ckpt.index size_local : 85618 size in s3 : 85618 create time local : 2023-03-03 12:26:17 create time in s3 : 2023-02-22 12:54:30 model_checkpoint.ckpt.index already exist and didn't need to update /data/models_weight/learn_papier_mixte_centre_tf_b4_220223_3/model_weights.h5 size_local : 71289448 size in s3 : 71289448 create time local : 2023-03-03 12:26:18 create time in s3 : 2023-02-22 12:54:17 model_weights.h5 already exist and didn't need to update ERROR in datou_step_exec, will save and exit ! assertion failed: [0] [Op:Assert] name: EagerVariableNameReuse File "/home/admin/workarea/git/Velours/python/mtr/datou/datou_lib.py", line 2339, in datou_exec output = datou_step_exec(sNext, args, cache, context, map_info, verbose, mtr_user_id) File "/home/admin/workarea/git/Velours/python/mtr/datou/datou_lib.py", line 2533, in datou_step_exec return lib_process.datou_step_tfhub2(param, json_param, args, cache, context, map_info, verbose) File "/home/admin/workarea/git/Velours/python/mtr/datou/lib_step_exec/lib_step_process.py", line 3139, in datou_step_tfhub2 this_model = model_evaluator(model_name, model_type=model_type, fc_size=fc_size,use_multi_inputs=use_multi_inputs) File "/home/admin/workarea/git/Velours/python/mtr/tfhub2/evaluate.py", line 156, in __init__ self.model, _, _ = create_tfhub_model(module_handle=self.tfhub_module, File "/home/admin/workarea/git/Velours/python/mtr/tfhub2/evaluate.py", line 77, in create_tfhub_model hub.KerasLayer(module_handle, trainable=do_fine_tuning, name="module"), File "/home/admin/.local/lib/python3.8/site-packages/tensorflow_hub/keras_layer.py", line 152, in __init__ self._func = load_module(handle, tags, self._load_options) File "/home/admin/.local/lib/python3.8/site-packages/tensorflow_hub/keras_layer.py", line 421, in load_module return module_v2.load(handle, tags=tags, options=set_load_options) File "/home/admin/.local/lib/python3.8/site-packages/tensorflow_hub/module_v2.py", line 106, in load obj = tf.compat.v1.saved_model.load_v2(module_path, tags=tags) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 578, in load return load_internal(export_dir, tags) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 602, in load_internal loader = loader_cls(object_graph_proto, File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 123, in __init__ self._load_all() File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 134, in _load_all self._load_nodes() File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 264, in _load_nodes node, setter = self._recreate(proto, node_id) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 370, in _recreate return factory[kind]() File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 363, in "variable": lambda: self._recreate_variable(proto.variable), File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 426, in _recreate_variable return variables.Variable( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/variables.py", line 261, in __call__ return cls._variable_v2_call(*args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/variables.py", line 243, in _variable_v2_call return previous_getter( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/variables.py", line 66, in getter return captured_getter(captured_previous, **kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 418, in uninitialized_variable_creator return resource_variable_ops.UninitializedVariable(**kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/variables.py", line 263, in __call__ return super(VariableMetaclass, cls).__call__(*args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/resource_variable_ops.py", line 1795, in __init__ handle = _variable_handle_from_shape_and_dtype( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/resource_variable_ops.py", line 174, in _variable_handle_from_shape_and_dtype gen_logging_ops._assert( # pylint: disable=protected-access File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/gen_logging_ops.py", line 55, in _assert _ops.raise_from_not_ok_status(e, name) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/ops.py", line 6653, in raise_from_not_ok_status six.raise_from(core._status_to_exception(e.code, message), None) File "", line 3, in raise_from [1385898658, 1385898622, 1385898582, 1385898539, 1385898498, 1385898459, 1385897627, 1385897622, 1385897613, 1385897612, 1385897611, 1385897609, 1385897405, 1385897404, 1385897379] begin to insert list_values into mtr_datou_result : length of list_values in save_final : 15 time used for this insertion : 0.016608238220214844 save_final ERROR in last step tfhub_classification2, assertion failed: [0] [Op:Assert] name: EagerVariableNameReuse time spend for datou_step_exec : 21.027831554412842 time spend to save output : 0.02842402458190918 total time spend for step 5 : 21.05625557899475 need to delete datou_research and reload, so keep current state 1 caffe_path_current : About to save ! 2 After save, about to update current ! 264.65user 131.10system 6:45.65elapsed 97%CPU (0avgtext+0avgdata 4597652maxresident)k 5837200inputs+275584outputs (74214major+14663245minor)pagefaults 0swaps