python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 3459 -l 10 ' -s datou_current_3459 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2861688 load datou : 3459 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2868 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5288 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 we have a portfolio with more photos than limit : 13>10 please execute split_portfolio.py -i 20879433 -l 10 size over we load limit photo not treated list_input_json: [] Current got : datou_id : 3459, datou_cur_ids : ['2619516'] with mtr_portfolio_ids : ['20879433'] and first list_photo_ids : [] new path : /proc/2861688/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, thcl, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBHTTP Error 503: Service Unavailable can't download the photo : 1339878612 complete_path : /home/admin/workarea/git/Velours/python/temp/1740643947_2861688_1339878612_42eb3f64729235a26e3111519bb5b4ca.jpg Fwe have missing 1 photos in the step downloads : photo missing : [1339878612] try to delete the photos missing in DB Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1339878612 length of list_filenames : 9 ; length of list_pids : 9 ; length of list_args : 9 time to download the photos : 26.136370420455933 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Thu Feb 27 09:12:53 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 3000 l 3632 free memory gpu now : 1958 wait 20 seconds l 3637 free memory gpu now : 1958 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-27 09:13:15.912551: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-27 09:13:15.939068: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-27 09:13:15.940866: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f65cc000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-27 09:13:15.940918: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-27 09:13:15.943502: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-27 09:13:16.157062: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5de8660 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-27 09:13:16.157108: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-27 09:13:16.158039: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-27 09:13:16.158443: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-27 09:13:16.161666: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-27 09:13:16.167960: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-27 09:13:16.168374: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-27 09:13:16.172646: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-27 09:13:16.174309: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-27 09:13:16.181500: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-27 09:13:16.182447: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-27 09:13:16.182525: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-27 09:13:16.183046: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-27 09:13:16.183062: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-27 09:13:16.183071: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-27 09:13:16.183862: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1802 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-27 09:13:16.460584: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-27 09:13:16.460717: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-27 09:13:16.460746: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-27 09:13:16.460772: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-27 09:13:16.460796: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-27 09:13:16.460821: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-27 09:13:16.460845: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-27 09:13:16.460869: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-27 09:13:16.462045: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-27 09:13:16.463257: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-27 09:13:16.463288: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-27 09:13:16.463303: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-27 09:13:16.463317: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-27 09:13:16.463331: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-27 09:13:16.463345: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-27 09:13:16.463359: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-27 09:13:16.463373: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-27 09:13:16.464103: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-27 09:13:16.464135: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-27 09:13:16.464143: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-27 09:13:16.464150: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-27 09:13:16.464858: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1802 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-27 09:13:26.115217: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-27 09:13:26.295470: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-27 09:13:27.605357: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-27 09:13:27.605422: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-27 09:13:27.611836: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-27 09:13:27.611860: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-27 09:13:27.664032: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-27 09:13:27.664086: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-27 09:13:27.705670: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.09GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-27 09:13:27.705700: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.09GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-27 09:13:27.756816: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.15GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-27 09:13:27.756852: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.15GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-27 09:13:27.758503: W tensorflow/core/common_runtime/bfc_allocator.cc:311] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature. 2025-02-27 09:13:27.775725: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.776220: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.13G (1219166208 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.776719: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.806961: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.807471: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.809090: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.809590: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.815732: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.816229: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.817853: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.818346: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.823989: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.824484: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.826164: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.826662: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.852623: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.853127: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.853616: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.854106: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.857471: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.857980: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.873045: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.873547: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.874036: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.874525: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.886570: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.887071: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.887563: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.888054: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.892160: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.892655: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.897047: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.897541: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.909211: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.909709: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.913681: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.914179: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.914823: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.915353: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.925733: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.926295: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.926846: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.927412: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.927921: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:27.928412: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:28.010917: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:28.011485: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:28.028538: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:28.029073: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:28.029577: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:28.030067: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:28.034029: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:28.034525: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:28.035032: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:28.035526: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:28.036401: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:28.036416: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-02-27 09:13:28.046700: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:28.047212: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:28.056079: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:28.056578: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:28.057078: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:28.057571: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:28.058070: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-27 09:13:28.058561: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1354629120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 9 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 24 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 24 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 40 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 24 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 19 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 17 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 22 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 24 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 2 Detection mask done ! Trying to reset tf kernel 2863648 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 986 tf kernel not reseted sub process len(results) : 9 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 9 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 2179 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.00022530555725097656 nb_pixel_total : 4416 time to create 1 rle with old method : 0.005363941192626953 length of segment : 169 time for calcul the mask position with numpy : 0.010969877243041992 nb_pixel_total : 287879 time to create 1 rle with new method : 0.02469325065612793 length of segment : 854 time for calcul the mask position with numpy : 8.96453857421875e-05 nb_pixel_total : 3120 time to create 1 rle with old method : 0.004151105880737305 length of segment : 59 time for calcul the mask position with numpy : 0.0002956390380859375 nb_pixel_total : 14967 time to create 1 rle with old method : 0.01690530776977539 length of segment : 215 time for calcul the mask position with numpy : 9.226799011230469e-05 nb_pixel_total : 1791 time to create 1 rle with old method : 0.002251148223876953 length of segment : 78 time for calcul the mask position with numpy : 0.049994707107543945 nb_pixel_total : 1274077 time to create 1 rle with new method : 0.9014945030212402 length of segment : 2547 time for calcul the mask position with numpy : 0.0014562606811523438 nb_pixel_total : 114881 time to create 1 rle with old method : 0.12793374061584473 length of segment : 454 time for calcul the mask position with numpy : 0.0019507408142089844 nb_pixel_total : 163191 time to create 1 rle with new method : 0.0044291019439697266 length of segment : 477 time for calcul the mask position with numpy : 0.0012454986572265625 nb_pixel_total : 50276 time to create 1 rle with old method : 0.06252503395080566 length of segment : 365 time for calcul the mask position with numpy : 0.00014209747314453125 nb_pixel_total : 4654 time to create 1 rle with old method : 0.005605459213256836 length of segment : 94 time for calcul the mask position with numpy : 0.0004627704620361328 nb_pixel_total : 31138 time to create 1 rle with old method : 0.03563547134399414 length of segment : 129 time for calcul the mask position with numpy : 0.0012454986572265625 nb_pixel_total : 54547 time to create 1 rle with old method : 0.07343840599060059 length of segment : 462 time for calcul the mask position with numpy : 0.00029087066650390625 nb_pixel_total : 10308 time to create 1 rle with old method : 0.013402938842773438 length of segment : 128 time for calcul the mask position with numpy : 0.00010943412780761719 nb_pixel_total : 4081 time to create 1 rle with old method : 0.005263090133666992 length of segment : 80 time for calcul the mask position with numpy : 0.003649473190307617 nb_pixel_total : 178073 time to create 1 rle with new method : 0.010717630386352539 length of segment : 908 time for calcul the mask position with numpy : 8.916854858398438e-05 nb_pixel_total : 1486 time to create 1 rle with old method : 0.0021514892578125 length of segment : 47 time for calcul the mask position with numpy : 0.0018184185028076172 nb_pixel_total : 141482 time to create 1 rle with old method : 0.16335439682006836 length of segment : 632 time for calcul the mask position with numpy : 0.00035381317138671875 nb_pixel_total : 14696 time to create 1 rle with old method : 0.02051687240600586 length of segment : 282 time for calcul the mask position with numpy : 0.0010764598846435547 nb_pixel_total : 27893 time to create 1 rle with old method : 0.03298163414001465 length of segment : 334 time for calcul the mask position with numpy : 0.000827789306640625 nb_pixel_total : 26117 time to create 1 rle with old method : 0.03055286407470703 length of segment : 217 time for calcul the mask position with numpy : 0.0002841949462890625 nb_pixel_total : 2550 time to create 1 rle with old method : 0.004262447357177734 length of segment : 45 time for calcul the mask position with numpy : 0.0005288124084472656 nb_pixel_total : 7122 time to create 1 rle with old method : 0.008750677108764648 length of segment : 267 time for calcul the mask position with numpy : 0.0012423992156982422 nb_pixel_total : 42511 time to create 1 rle with old method : 0.049637556076049805 length of segment : 241 time for calcul the mask position with numpy : 0.03396463394165039 nb_pixel_total : 807707 time to create 1 rle with new method : 0.09732866287231445 length of segment : 1318 time for calcul the mask position with numpy : 0.0001380443572998047 nb_pixel_total : 4332 time to create 1 rle with old method : 0.0056116580963134766 length of segment : 55 time for calcul the mask position with numpy : 0.003433704376220703 nb_pixel_total : 150445 time to create 1 rle with new method : 0.007627725601196289 length of segment : 659 time for calcul the mask position with numpy : 0.0026082992553710938 nb_pixel_total : 91058 time to create 1 rle with old method : 0.10670924186706543 length of segment : 374 time for calcul the mask position with numpy : 0.0012392997741699219 nb_pixel_total : 61740 time to create 1 rle with old method : 0.0717933177947998 length of segment : 264 time for calcul the mask position with numpy : 0.010638713836669922 nb_pixel_total : 456423 time to create 1 rle with new method : 0.0330657958984375 length of segment : 784 time for calcul the mask position with numpy : 0.0005609989166259766 nb_pixel_total : 8837 time to create 1 rle with old method : 0.010355710983276367 length of segment : 209 time for calcul the mask position with numpy : 0.0005807876586914062 nb_pixel_total : 22084 time to create 1 rle with old method : 0.026988983154296875 length of segment : 233 time for calcul the mask position with numpy : 0.003832101821899414 nb_pixel_total : 188505 time to create 1 rle with new method : 0.007157802581787109 length of segment : 579 time for calcul the mask position with numpy : 0.008494377136230469 nb_pixel_total : 444221 time to create 1 rle with new method : 0.01642584800720215 length of segment : 614 time for calcul the mask position with numpy : 0.0008516311645507812 nb_pixel_total : 29636 time to create 1 rle with old method : 0.0359044075012207 length of segment : 239 time for calcul the mask position with numpy : 0.0005581378936767578 nb_pixel_total : 17042 time to create 1 rle with old method : 0.020430326461791992 length of segment : 232 time for calcul the mask position with numpy : 0.004045724868774414 nb_pixel_total : 202359 time to create 1 rle with new method : 0.00982666015625 length of segment : 619 time for calcul the mask position with numpy : 0.00022745132446289062 nb_pixel_total : 4655 time to create 1 rle with old method : 0.00573420524597168 length of segment : 116 time for calcul the mask position with numpy : 0.005433320999145508 nb_pixel_total : 164611 time to create 1 rle with new method : 0.006733894348144531 length of segment : 576 time for calcul the mask position with numpy : 0.0001938343048095703 nb_pixel_total : 1161 time to create 1 rle with old method : 0.0015664100646972656 length of segment : 59 time for calcul the mask position with numpy : 0.0013422966003417969 nb_pixel_total : 17197 time to create 1 rle with old method : 0.021719932556152344 length of segment : 261 time for calcul the mask position with numpy : 0.0003294944763183594 nb_pixel_total : 3539 time to create 1 rle with old method : 0.004412651062011719 length of segment : 81 time for calcul the mask position with numpy : 0.0005216598510742188 nb_pixel_total : 10613 time to create 1 rle with old method : 0.013178110122680664 length of segment : 137 time for calcul the mask position with numpy : 0.0015788078308105469 nb_pixel_total : 32613 time to create 1 rle with old method : 0.04000234603881836 length of segment : 211 time for calcul the mask position with numpy : 0.0007965564727783203 nb_pixel_total : 15335 time to create 1 rle with old method : 0.01895761489868164 length of segment : 202 time for calcul the mask position with numpy : 0.003414154052734375 nb_pixel_total : 101356 time to create 1 rle with old method : 0.11748266220092773 length of segment : 408 time for calcul the mask position with numpy : 0.001438140869140625 nb_pixel_total : 23281 time to create 1 rle with old method : 0.02848672866821289 length of segment : 275 time for calcul the mask position with numpy : 0.001796722412109375 nb_pixel_total : 48084 time to create 1 rle with old method : 0.059287071228027344 length of segment : 335 time for calcul the mask position with numpy : 0.0024652481079101562 nb_pixel_total : 22464 time to create 1 rle with old method : 0.03743124008178711 length of segment : 517 time for calcul the mask position with numpy : 0.0007982254028320312 nb_pixel_total : 15826 time to create 1 rle with old method : 0.020032405853271484 length of segment : 183 time for calcul the mask position with numpy : 0.0005850791931152344 nb_pixel_total : 14166 time to create 1 rle with old method : 0.016872167587280273 length of segment : 139 time for calcul the mask position with numpy : 0.0007662773132324219 nb_pixel_total : 18206 time to create 1 rle with old method : 0.030833721160888672 length of segment : 199 time for calcul the mask position with numpy : 0.0006330013275146484 nb_pixel_total : 16350 time to create 1 rle with old method : 0.019864559173583984 length of segment : 167 time for calcul the mask position with numpy : 0.0003535747528076172 nb_pixel_total : 2997 time to create 1 rle with old method : 0.003519296646118164 length of segment : 111 time for calcul the mask position with numpy : 0.0014591217041015625 nb_pixel_total : 23804 time to create 1 rle with old method : 0.028102874755859375 length of segment : 251 time for calcul the mask position with numpy : 0.00032901763916015625 nb_pixel_total : 5164 time to create 1 rle with old method : 0.0064318180084228516 length of segment : 107 time for calcul the mask position with numpy : 0.0069103240966796875 nb_pixel_total : 206759 time to create 1 rle with new method : 0.009215354919433594 length of segment : 557 time for calcul the mask position with numpy : 0.010716676712036133 nb_pixel_total : 282821 time to create 1 rle with new method : 0.015017509460449219 length of segment : 1045 time for calcul the mask position with numpy : 0.0003535747528076172 nb_pixel_total : 6035 time to create 1 rle with old method : 0.0075969696044921875 length of segment : 128 time for calcul the mask position with numpy : 0.005232572555541992 nb_pixel_total : 111652 time to create 1 rle with old method : 0.12590789794921875 length of segment : 500 time for calcul the mask position with numpy : 0.0007081031799316406 nb_pixel_total : 16319 time to create 1 rle with old method : 0.019809961318969727 length of segment : 147 time for calcul the mask position with numpy : 0.0003349781036376953 nb_pixel_total : 9235 time to create 1 rle with old method : 0.011878252029418945 length of segment : 112 time for calcul the mask position with numpy : 0.003210306167602539 nb_pixel_total : 62709 time to create 1 rle with old method : 0.07348775863647461 length of segment : 646 time for calcul the mask position with numpy : 0.0021414756774902344 nb_pixel_total : 42022 time to create 1 rle with old method : 0.04797935485839844 length of segment : 506 time for calcul the mask position with numpy : 0.0011060237884521484 nb_pixel_total : 31184 time to create 1 rle with old method : 0.036659955978393555 length of segment : 140 time for calcul the mask position with numpy : 0.0008440017700195312 nb_pixel_total : 5326 time to create 1 rle with old method : 0.0067119598388671875 length of segment : 214 time for calcul the mask position with numpy : 0.0013070106506347656 nb_pixel_total : 17147 time to create 1 rle with old method : 0.02046942710876465 length of segment : 295 time for calcul the mask position with numpy : 0.0034494400024414062 nb_pixel_total : 112792 time to create 1 rle with old method : 0.1371781826019287 length of segment : 489 time for calcul the mask position with numpy : 0.00023865699768066406 nb_pixel_total : 2082 time to create 1 rle with old method : 0.002563953399658203 length of segment : 72 time for calcul the mask position with numpy : 0.0005497932434082031 nb_pixel_total : 8714 time to create 1 rle with old method : 0.010559320449829102 length of segment : 122 time for calcul the mask position with numpy : 0.0011394023895263672 nb_pixel_total : 23513 time to create 1 rle with old method : 0.027925968170166016 length of segment : 285 time for calcul the mask position with numpy : 0.0004839897155761719 nb_pixel_total : 6549 time to create 1 rle with old method : 0.008115291595458984 length of segment : 152 time for calcul the mask position with numpy : 0.004704952239990234 nb_pixel_total : 112195 time to create 1 rle with old method : 0.13005971908569336 length of segment : 414 time for calcul the mask position with numpy : 0.005034208297729492 nb_pixel_total : 202622 time to create 1 rle with new method : 0.008863449096679688 length of segment : 561 time for calcul the mask position with numpy : 0.0009617805480957031 nb_pixel_total : 28771 time to create 1 rle with old method : 0.03481602668762207 length of segment : 242 time for calcul the mask position with numpy : 0.0004923343658447266 nb_pixel_total : 9675 time to create 1 rle with old method : 0.011939048767089844 length of segment : 166 time for calcul the mask position with numpy : 0.0003046989440917969 nb_pixel_total : 3661 time to create 1 rle with old method : 0.004509687423706055 length of segment : 149 time for calcul the mask position with numpy : 0.0002841949462890625 nb_pixel_total : 7561 time to create 1 rle with old method : 0.009217262268066406 length of segment : 73 time for calcul the mask position with numpy : 0.003058195114135742 nb_pixel_total : 85393 time to create 1 rle with old method : 0.10795807838439941 length of segment : 456 time for calcul the mask position with numpy : 0.0036954879760742188 nb_pixel_total : 99544 time to create 1 rle with old method : 0.11140322685241699 length of segment : 511 time for calcul the mask position with numpy : 0.00022792816162109375 nb_pixel_total : 2899 time to create 1 rle with old method : 0.003675699234008789 length of segment : 78 time for calcul the mask position with numpy : 0.002224445343017578 nb_pixel_total : 69635 time to create 1 rle with old method : 0.0794522762298584 length of segment : 377 time for calcul the mask position with numpy : 0.00026035308837890625 nb_pixel_total : 4228 time to create 1 rle with old method : 0.005370140075683594 length of segment : 99 time for calcul the mask position with numpy : 0.0011456012725830078 nb_pixel_total : 24850 time to create 1 rle with old method : 0.03086996078491211 length of segment : 245 time for calcul the mask position with numpy : 0.0012624263763427734 nb_pixel_total : 32002 time to create 1 rle with old method : 0.03714728355407715 length of segment : 315 time for calcul the mask position with numpy : 0.0002675056457519531 nb_pixel_total : 1320 time to create 1 rle with old method : 0.002401590347290039 length of segment : 69 time for calcul the mask position with numpy : 0.0010728836059570312 nb_pixel_total : 20933 time to create 1 rle with old method : 0.03482317924499512 length of segment : 201 time for calcul the mask position with numpy : 0.0037550926208496094 nb_pixel_total : 131158 time to create 1 rle with old method : 0.15866971015930176 length of segment : 513 time for calcul the mask position with numpy : 0.0038352012634277344 nb_pixel_total : 117351 time to create 1 rle with old method : 0.1365673542022705 length of segment : 617 time for calcul the mask position with numpy : 0.00012993812561035156 nb_pixel_total : 1469 time to create 1 rle with old method : 0.0020780563354492188 length of segment : 35 time for calcul the mask position with numpy : 0.00173187255859375 nb_pixel_total : 41791 time to create 1 rle with old method : 0.05295896530151367 length of segment : 356 time for calcul the mask position with numpy : 0.0001652240753173828 nb_pixel_total : 2964 time to create 1 rle with old method : 0.0036995410919189453 length of segment : 45 time for calcul the mask position with numpy : 0.0001704692840576172 nb_pixel_total : 3971 time to create 1 rle with old method : 0.0048160552978515625 length of segment : 60 time for calcul the mask position with numpy : 0.0019404888153076172 nb_pixel_total : 99862 time to create 1 rle with old method : 0.11687374114990234 length of segment : 360 time for calcul the mask position with numpy : 0.001251220703125 nb_pixel_total : 63624 time to create 1 rle with old method : 0.07617592811584473 length of segment : 350 time for calcul the mask position with numpy : 0.0012841224670410156 nb_pixel_total : 53252 time to create 1 rle with old method : 0.061965227127075195 length of segment : 519 time for calcul the mask position with numpy : 0.0028324127197265625 nb_pixel_total : 167364 time to create 1 rle with new method : 0.005033731460571289 length of segment : 771 time for calcul the mask position with numpy : 0.0017838478088378906 nb_pixel_total : 87495 time to create 1 rle with old method : 0.10319638252258301 length of segment : 681 time for calcul the mask position with numpy : 0.002354145050048828 nb_pixel_total : 79551 time to create 1 rle with old method : 0.09191036224365234 length of segment : 399 time for calcul the mask position with numpy : 0.00593256950378418 nb_pixel_total : 359464 time to create 1 rle with new method : 0.015105962753295898 length of segment : 826 time for calcul the mask position with numpy : 0.0034923553466796875 nb_pixel_total : 150047 time to create 1 rle with new method : 0.008652925491333008 length of segment : 1234 time for calcul the mask position with numpy : 0.0005240440368652344 nb_pixel_total : 21819 time to create 1 rle with old method : 0.025677919387817383 length of segment : 273 time for calcul the mask position with numpy : 0.0006885528564453125 nb_pixel_total : 21473 time to create 1 rle with old method : 0.024643421173095703 length of segment : 212 time for calcul the mask position with numpy : 0.00017905235290527344 nb_pixel_total : 4279 time to create 1 rle with old method : 0.005315542221069336 length of segment : 68 time for calcul the mask position with numpy : 0.0010383129119873047 nb_pixel_total : 27100 time to create 1 rle with old method : 0.03249406814575195 length of segment : 186 time for calcul the mask position with numpy : 0.0014171600341796875 nb_pixel_total : 61091 time to create 1 rle with old method : 0.0706181526184082 length of segment : 276 time for calcul the mask position with numpy : 0.000362396240234375 nb_pixel_total : 23927 time to create 1 rle with old method : 0.027515411376953125 length of segment : 175 time for calcul the mask position with numpy : 0.0015406608581542969 nb_pixel_total : 81575 time to create 1 rle with old method : 0.09467053413391113 length of segment : 397 time for calcul the mask position with numpy : 0.0008587837219238281 nb_pixel_total : 42085 time to create 1 rle with old method : 0.04668736457824707 length of segment : 333 time for calcul the mask position with numpy : 0.0010342597961425781 nb_pixel_total : 38543 time to create 1 rle with old method : 0.04600930213928223 length of segment : 291 time for calcul the mask position with numpy : 0.00028252601623535156 nb_pixel_total : 5103 time to create 1 rle with old method : 0.008652210235595703 length of segment : 163 time for calcul the mask position with numpy : 0.002160787582397461 nb_pixel_total : 78954 time to create 1 rle with old method : 0.09815287590026855 length of segment : 412 time for calcul the mask position with numpy : 0.0003497600555419922 nb_pixel_total : 13053 time to create 1 rle with old method : 0.01576709747314453 length of segment : 215 time for calcul the mask position with numpy : 0.0020503997802734375 nb_pixel_total : 102407 time to create 1 rle with old method : 0.11362862586975098 length of segment : 541 time for calcul the mask position with numpy : 0.00180816650390625 nb_pixel_total : 70708 time to create 1 rle with old method : 0.08231616020202637 length of segment : 306 time for calcul the mask position with numpy : 0.0008313655853271484 nb_pixel_total : 20603 time to create 1 rle with old method : 0.02402043342590332 length of segment : 316 time for calcul the mask position with numpy : 0.000762939453125 nb_pixel_total : 14365 time to create 1 rle with old method : 0.01706528663635254 length of segment : 227 time for calcul the mask position with numpy : 0.0016472339630126953 nb_pixel_total : 61038 time to create 1 rle with old method : 0.07194113731384277 length of segment : 436 time for calcul the mask position with numpy : 0.001177072525024414 nb_pixel_total : 49707 time to create 1 rle with old method : 0.055193424224853516 length of segment : 387 time for calcul the mask position with numpy : 0.0002124309539794922 nb_pixel_total : 4103 time to create 1 rle with old method : 0.0052661895751953125 length of segment : 65 time for calcul the mask position with numpy : 0.0019197463989257812 nb_pixel_total : 51231 time to create 1 rle with old method : 0.06319379806518555 length of segment : 392 time for calcul the mask position with numpy : 0.00013875961303710938 nb_pixel_total : 1013 time to create 1 rle with old method : 0.0014047622680664062 length of segment : 33 time for calcul the mask position with numpy : 0.002851724624633789 nb_pixel_total : 72660 time to create 1 rle with old method : 0.08614063262939453 length of segment : 631 time for calcul the mask position with numpy : 0.00024127960205078125 nb_pixel_total : 4531 time to create 1 rle with old method : 0.005279541015625 length of segment : 74 time for calcul the mask position with numpy : 0.0013475418090820312 nb_pixel_total : 50039 time to create 1 rle with old method : 0.05798792839050293 length of segment : 301 time for calcul the mask position with numpy : 0.001125335693359375 nb_pixel_total : 44390 time to create 1 rle with old method : 0.052783966064453125 length of segment : 289 time for calcul the mask position with numpy : 0.001157522201538086 nb_pixel_total : 46696 time to create 1 rle with old method : 0.05125784873962402 length of segment : 321 time for calcul the mask position with numpy : 0.0012137889862060547 nb_pixel_total : 36982 time to create 1 rle with old method : 0.041466474533081055 length of segment : 369 time for calcul the mask position with numpy : 0.0010859966278076172 nb_pixel_total : 38068 time to create 1 rle with old method : 0.043918609619140625 length of segment : 201 time for calcul the mask position with numpy : 0.0005726814270019531 nb_pixel_total : 15254 time to create 1 rle with old method : 0.018018007278442383 length of segment : 191 time for calcul the mask position with numpy : 0.005456447601318359 nb_pixel_total : 156343 time to create 1 rle with new method : 0.013183116912841797 length of segment : 1054 time for calcul the mask position with numpy : 0.0006392002105712891 nb_pixel_total : 10610 time to create 1 rle with old method : 0.013108491897583008 length of segment : 154 time for calcul the mask position with numpy : 0.0030112266540527344 nb_pixel_total : 97531 time to create 1 rle with old method : 0.10829901695251465 length of segment : 560 time for calcul the mask position with numpy : 0.001163482666015625 nb_pixel_total : 46864 time to create 1 rle with old method : 0.055197715759277344 length of segment : 268 time for calcul the mask position with numpy : 0.0007343292236328125 nb_pixel_total : 18973 time to create 1 rle with old method : 0.0233767032623291 length of segment : 264 time for calcul the mask position with numpy : 0.0008878707885742188 nb_pixel_total : 26592 time to create 1 rle with old method : 0.03165721893310547 length of segment : 111 time for calcul the mask position with numpy : 0.0008249282836914062 nb_pixel_total : 48014 time to create 1 rle with old method : 0.05627155303955078 length of segment : 328 time for calcul the mask position with numpy : 9.846687316894531e-05 nb_pixel_total : 2336 time to create 1 rle with old method : 0.003056764602661133 length of segment : 40 time for calcul the mask position with numpy : 0.0024514198303222656 nb_pixel_total : 50271 time to create 1 rle with old method : 0.0603792667388916 length of segment : 514 time for calcul the mask position with numpy : 0.0006840229034423828 nb_pixel_total : 6992 time to create 1 rle with old method : 0.008348941802978516 length of segment : 251 time for calcul the mask position with numpy : 0.0009438991546630859 nb_pixel_total : 49670 time to create 1 rle with old method : 0.0568239688873291 length of segment : 312 time for calcul the mask position with numpy : 0.0010306835174560547 nb_pixel_total : 24113 time to create 1 rle with old method : 0.029572010040283203 length of segment : 296 time for calcul the mask position with numpy : 0.0026657581329345703 nb_pixel_total : 107904 time to create 1 rle with old method : 0.12142539024353027 length of segment : 524 time for calcul the mask position with numpy : 0.0007812976837158203 nb_pixel_total : 36372 time to create 1 rle with old method : 0.0430757999420166 length of segment : 495 time for calcul the mask position with numpy : 0.0005791187286376953 nb_pixel_total : 8334 time to create 1 rle with old method : 0.010529279708862305 length of segment : 237 time for calcul the mask position with numpy : 0.0006952285766601562 nb_pixel_total : 15698 time to create 1 rle with old method : 0.018141984939575195 length of segment : 233 time for calcul the mask position with numpy : 0.0022008419036865234 nb_pixel_total : 83180 time to create 1 rle with old method : 0.09981346130371094 length of segment : 308 time for calcul the mask position with numpy : 0.0006916522979736328 nb_pixel_total : 15764 time to create 1 rle with old method : 0.02015995979309082 length of segment : 139 time for calcul the mask position with numpy : 0.0015218257904052734 nb_pixel_total : 47760 time to create 1 rle with old method : 0.05448317527770996 length of segment : 321 time for calcul the mask position with numpy : 0.0011494159698486328 nb_pixel_total : 33081 time to create 1 rle with old method : 0.036950111389160156 length of segment : 263 time for calcul the mask position with numpy : 0.0016849040985107422 nb_pixel_total : 22705 time to create 1 rle with old method : 0.02675461769104004 length of segment : 208 time for calcul the mask position with numpy : 0.0011134147644042969 nb_pixel_total : 34237 time to create 1 rle with old method : 0.03823065757751465 length of segment : 403 time for calcul the mask position with numpy : 0.0007801055908203125 nb_pixel_total : 26570 time to create 1 rle with old method : 0.0313260555267334 length of segment : 214 time for calcul the mask position with numpy : 0.0017490386962890625 nb_pixel_total : 50829 time to create 1 rle with old method : 0.060575246810913086 length of segment : 454 time for calcul the mask position with numpy : 0.0012052059173583984 nb_pixel_total : 40701 time to create 1 rle with old method : 0.045625925064086914 length of segment : 220 time for calcul the mask position with numpy : 0.005999326705932617 nb_pixel_total : 256725 time to create 1 rle with new method : 0.008529186248779297 length of segment : 753 time for calcul the mask position with numpy : 0.00026535987854003906 nb_pixel_total : 4041 time to create 1 rle with old method : 0.004862070083618164 length of segment : 105 time for calcul the mask position with numpy : 0.001056671142578125 nb_pixel_total : 62553 time to create 1 rle with old method : 0.07546496391296387 length of segment : 417 time for calcul the mask position with numpy : 0.0012927055358886719 nb_pixel_total : 28803 time to create 1 rle with old method : 0.0341038703918457 length of segment : 396 time for calcul the mask position with numpy : 0.0017917156219482422 nb_pixel_total : 48482 time to create 1 rle with old method : 0.06050443649291992 length of segment : 319 time for calcul the mask position with numpy : 0.00024890899658203125 nb_pixel_total : 2869 time to create 1 rle with old method : 0.0035021305084228516 length of segment : 69 time for calcul the mask position with numpy : 0.00115966796875 nb_pixel_total : 19396 time to create 1 rle with old method : 0.024053335189819336 length of segment : 257 time for calcul the mask position with numpy : 0.00018024444580078125 nb_pixel_total : 2577 time to create 1 rle with old method : 0.0032265186309814453 length of segment : 97 time for calcul the mask position with numpy : 0.0005257129669189453 nb_pixel_total : 27500 time to create 1 rle with old method : 0.03250885009765625 length of segment : 212 time for calcul the mask position with numpy : 0.19734764099121094 nb_pixel_total : 3092770 time to create 1 rle with new method : 0.379744291305542 length of segment : 3104 time for calcul the mask position with numpy : 0.0011985301971435547 nb_pixel_total : 67452 time to create 1 rle with old method : 0.0791313648223877 length of segment : 337 time spent for convertir_results : 16.81140661239624 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 165 chid ids of type : 3663 Number RLEs to save : 58048 save missing photos in datou_result : time spend for datou_step_exec : 82.85137963294983 time spend to save output : 3.225083589553833 total time spend for step 1 : 86.07646322250366 step2:crop_condition Thu Feb 27 09:14:19 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 9 ! batch 1 Loaded 165 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 20 About to insert : list_path_to_insert length 20 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 55 About to insert : list_path_to_insert length 55 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1339879029, 1339879005, 1339878979, 1339878976, 1339878973, 1339878960, 1339878953, 1339878950, 1339878670] Looping around the photos to save general results len do output : 99 /-3688488779Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488800Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488794Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488795Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488803Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488791Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488821Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488830Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488841Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488843Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488857Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488864Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488859Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488863Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488875Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488879Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488876Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488874Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488898Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488895Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488765Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488809Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488827Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488909Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488903Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488759Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488780Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488820Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488826Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488888Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488887Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488756Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488757Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488760Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488752Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488763Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488755Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488771Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488784Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488782Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488776Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488796Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488797Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488818Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488793Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488804Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488807Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488786Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488815Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488812Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488802Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488810Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488808Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488829Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488835Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488836Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488831Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488844Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488848Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488846Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488842Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488845Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488847Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488853Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488852Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488849Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488865Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488870Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488861Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488856Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488866Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488883Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488880Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488886Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488881Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488890Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488884Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488882Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488894Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488899Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488897Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488896Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488905Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488906Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488907Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488913Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488761Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488774Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488775Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488788Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488792Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488873Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488900Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488893Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488838Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488832Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488862Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488902Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688488911Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339879029', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339879005', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878979', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878976', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878973', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878960', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878953', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878950', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878670', None, None, None, None, None, '2619516') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 306 time used for this insertion : 0.03364396095275879 save_final save missing photos in datou_result : time spend for datou_step_exec : 38.499484062194824 time spend to save output : 0.03688931465148926 total time spend for step 2 : 38.53637337684631 step3:thcl Thu Feb 27 09:14:57 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 2 we are using the classfication for multi_thcl [2456, 2868] time to import caffe and check if the image exist : 0.013629913330078125 time to convert the images to numpy array : 0.06650495529174805 time to import caffe and check if the image exist : 0.017331600189208984 time to convert the images to numpy array : 0.06736350059509277 time to import caffe and check if the image exist : 0.01148676872253418 time to convert the images to numpy array : 0.0966942310333252 time to import caffe and check if the image exist : 0.009342193603515625 time to convert the images to numpy array : 0.44237685203552246 time to import caffe and check if the image exist : 0.016286611557006836 time to convert the images to numpy array : 0.43810462951660156 time to import caffe and check if the image exist : 0.012303352355957031 time to convert the images to numpy array : 0.46149539947509766 time to import caffe and check if the image exist : 0.012792348861694336 time to convert the images to numpy array : 0.4615797996520996 time to import caffe and check if the image exist : 0.009727954864501953 time to convert the images to numpy array : 0.47609806060791016 time to import caffe and check if the image exist : 0.014967918395996094 time to convert the images to numpy array : 0.47613072395324707 time to import caffe and check if the image exist : 0.012534618377685547 time to convert the images to numpy array : 0.4897472858428955 total time to convert the images to numpy array : 0.7266240119934082 list photo_ids error: [] list photo_ids correct : [-3688488792, -3688488873, -3688488900, -3688488893, -3688488838, -3688488832, -3688488862, -3688488902, -3688488911, -3688488857, -3688488864, -3688488859, -3688488863, -3688488875, -3688488879, -3688488876, -3688488874, -3688488898, -3688488895, -3688488779, -3688488800, -3688488794, -3688488795, -3688488803, -3688488791, -3688488821, -3688488830, -3688488841, -3688488843, -3688488866, -3688488883, -3688488880, -3688488886, -3688488881, -3688488890, -3688488884, -3688488882, -3688488894, -3688488899, -3688488897, -3688488896, -3688488905, -3688488906, -3688488907, -3688488913, -3688488761, -3688488774, -3688488775, -3688488788, -3688488776, -3688488796, -3688488797, -3688488818, -3688488793, -3688488804, -3688488807, -3688488786, -3688488815, -3688488812, -3688488765, -3688488809, -3688488827, -3688488909, -3688488903, -3688488759, -3688488780, -3688488820, -3688488826, -3688488888, -3688488842, -3688488845, -3688488847, -3688488853, -3688488852, -3688488849, -3688488865, -3688488870, -3688488861, -3688488856, -3688488887, -3688488756, -3688488757, -3688488760, -3688488752, -3688488763, -3688488755, -3688488771, -3688488784, -3688488782, -3688488802, -3688488810, -3688488808, -3688488829, -3688488835, -3688488836, -3688488831, -3688488844, -3688488848, -3688488846] number of photos to traite : 99 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 2500 l 3632 free memory gpu now : 1380 wait 20 seconds l 3637 free memory gpu now : 1380 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 2500 l 3632 free memory gpu now : 1960 wait 20 seconds l 3637 free memory gpu now : 1960 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.0105020999908447 time used to do the prediction : 0.31845855712890625 we don't save the descriptors for this thcl 2456 tagging for thcl : 2868 To do loadFromThcl(), then load ParamDescType : thcl2868 thcls : [{'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5288 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) To loadFromThcl() : net_5288 begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 2500 l 3632 free memory gpu now : 1958 wait 20 seconds l 3637 free memory gpu now : 1958 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_papier_nantes_300421 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_papier_nantes_300421 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_papier_nantes_300421 /data/models_weight/learn_papier_nantes_300421/caffemodel size_local : 44791983 size in s3 : 44791983 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:13 caffemodel already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/deploy.prototxt size_local : 17255 size in s3 : 17255 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:12 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:14 mean.npy already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/synset_words.txt size_local : 331 size in s3 : 331 create time local : 2021-08-09 05:56:00 create time in s3 : 2021-08-06 19:22:12 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_papier_nantes_300421/deploy.prototxt caffemodel_filename : /data/models_weight/learn_papier_nantes_300421/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 2500 l 3632 free memory gpu now : 1739 wait 20 seconds l 3637 free memory gpu now : 1739 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.1804044246673584 time used to do the prediction : 0.27837157249450684 we don't save the descriptors for this thcl 2868 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 2.6941299438476562e-05 save missing photos in datou_result : time spend for datou_step_exec : 90.56070327758789 time spend to save output : 0.0007302761077880859 total time spend for step 3 : 90.56143355369568 step4:merge_mask_thcl_custom Thu Feb 27 09:16:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 165 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1339879029, 1339879005, 1339878979, 1339878976, 1339878973, 1339878960, 1339878953, 1339878950, 1339878670] Looping around the photos to save general results len do output : 9 /1339879029Didn't retrieve data .Didn't retrieve data . /1339879005Didn't retrieve data .Didn't retrieve data . /1339878979Didn't retrieve data .Didn't retrieve data . /1339878976Didn't retrieve data .Didn't retrieve data . /1339878973Didn't retrieve data .Didn't retrieve data . /1339878960Didn't retrieve data .Didn't retrieve data . /1339878953Didn't retrieve data .Didn't retrieve data . /1339878950Didn't retrieve data .Didn't retrieve data . /1339878670Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339879029', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339879005', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878979', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878976', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878973', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878960', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878953', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878950', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878670', None, None, None, None, None, '2619516') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 27 time used for this insertion : 0.01367950439453125 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.06173539161682129 time spend to save output : 0.014223337173461914 total time spend for step 4 : 0.0759587287902832 step5:rle_unique_nms_with_priority Thu Feb 27 09:16:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 16 nb_hashtags : 5 time to prepare the origin masks : 13.111429452896118 time for calcul the mask position with numpy : 0.358351469039917 nb_pixel_total : 4919459 time to create 1 rle with new method : 0.6718499660491943 time for calcul the mask position with numpy : 0.036458492279052734 nb_pixel_total : 47804 time to create 1 rle with old method : 0.05797123908996582 time for calcul the mask position with numpy : 0.03592848777770996 nb_pixel_total : 49787 time to create 1 rle with old method : 0.05718874931335449 time for calcul the mask position with numpy : 0.03466486930847168 nb_pixel_total : 10281 time to create 1 rle with old method : 0.013293027877807617 time for calcul the mask position with numpy : 0.03476548194885254 nb_pixel_total : 14660 time to create 1 rle with old method : 0.016707181930541992 time for calcul the mask position with numpy : 0.034852027893066406 nb_pixel_total : 34942 time to create 1 rle with old method : 0.04440784454345703 time for calcul the mask position with numpy : 0.036412715911865234 nb_pixel_total : 101531 time to create 1 rle with old method : 0.13978838920593262 time for calcul the mask position with numpy : 0.04489850997924805 nb_pixel_total : 4635 time to create 1 rle with old method : 0.00568842887878418 time for calcul the mask position with numpy : 0.03948974609375 nb_pixel_total : 13899 time to create 1 rle with old method : 0.01678180694580078 time for calcul the mask position with numpy : 0.05277442932128906 nb_pixel_total : 1293347 time to create 1 rle with new method : 0.5297479629516602 time for calcul the mask position with numpy : 0.036366939544677734 nb_pixel_total : 1776 time to create 1 rle with old method : 0.0021545886993408203 time for calcul the mask position with numpy : 0.03460383415222168 nb_pixel_total : 3103 time to create 1 rle with old method : 0.00345611572265625 time for calcul the mask position with numpy : 0.03860878944396973 nb_pixel_total : 287620 time to create 1 rle with new method : 1.0594406127929688 time for calcul the mask position with numpy : 0.033217430114746094 nb_pixel_total : 30969 time to create 1 rle with old method : 0.037490129470825195 time for calcul the mask position with numpy : 0.03515028953552246 nb_pixel_total : 54243 time to create 1 rle with old method : 0.06260967254638672 time for calcul the mask position with numpy : 0.03417611122131348 nb_pixel_total : 4418 time to create 1 rle with old method : 0.005198240280151367 time for calcul the mask position with numpy : 0.035890817642211914 nb_pixel_total : 177766 time to create 1 rle with new method : 0.9460146427154541 create new chi : 4.729695796966553 time to delete rle : 0.01531362533569336 batch 1 Loaded 17 chid ids of type : 3726 Number RLEs to save : 15436 TO DO : save crop sub photo not yet done ! save time : 0.8662760257720947 nb_obj : 19 nb_hashtags : 7 time to prepare the origin masks : 12.684794425964355 time for calcul the mask position with numpy : 0.34563398361206055 nb_pixel_total : 4856307 time to create 1 rle with new method : 0.707855224609375 time for calcul the mask position with numpy : 0.021158456802368164 nb_pixel_total : 21882 time to create 1 rle with old method : 0.025572538375854492 time for calcul the mask position with numpy : 0.020821571350097656 nb_pixel_total : 4649 time to create 1 rle with old method : 0.00545191764831543 time for calcul the mask position with numpy : 0.02118659019470215 nb_pixel_total : 16854 time to create 1 rle with old method : 0.019762039184570312 time for calcul the mask position with numpy : 0.021190643310546875 nb_pixel_total : 29587 time to create 1 rle with old method : 0.034287452697753906 time for calcul the mask position with numpy : 0.02909398078918457 nb_pixel_total : 236197 time to create 1 rle with new method : 0.503321647644043 time for calcul the mask position with numpy : 0.01995253562927246 nb_pixel_total : 42048 time to create 1 rle with old method : 0.04668593406677246 time for calcul the mask position with numpy : 0.020746231079101562 nb_pixel_total : 755 time to create 1 rle with old method : 0.001256704330444336 time for calcul the mask position with numpy : 0.021294593811035156 nb_pixel_total : 8817 time to create 1 rle with old method : 0.010256290435791016 time for calcul the mask position with numpy : 0.020760774612426758 nb_pixel_total : 61669 time to create 1 rle with old method : 0.07013201713562012 time for calcul the mask position with numpy : 0.025229692459106445 nb_pixel_total : 202309 time to create 1 rle with new method : 0.5419220924377441 time for calcul the mask position with numpy : 0.03791689872741699 nb_pixel_total : 456043 time to create 1 rle with new method : 0.5144281387329102 time for calcul the mask position with numpy : 0.03553318977355957 nb_pixel_total : 4318 time to create 1 rle with old method : 0.005178689956665039 time for calcul the mask position with numpy : 0.04555940628051758 nb_pixel_total : 803875 time to create 1 rle with new method : 0.5972771644592285 time for calcul the mask position with numpy : 0.03381538391113281 nb_pixel_total : 150380 time to create 1 rle with new method : 0.5250113010406494 time for calcul the mask position with numpy : 0.028877973556518555 nb_pixel_total : 90999 time to create 1 rle with old method : 0.10463595390319824 time for calcul the mask position with numpy : 0.025908946990966797 nb_pixel_total : 7127 time to create 1 rle with old method : 0.008601665496826172 time for calcul the mask position with numpy : 0.020673751831054688 nb_pixel_total : 2524 time to create 1 rle with old method : 0.0029811859130859375 time for calcul the mask position with numpy : 0.02178788185119629 nb_pixel_total : 26036 time to create 1 rle with old method : 0.0296933650970459 time for calcul the mask position with numpy : 0.02205348014831543 nb_pixel_total : 27864 time to create 1 rle with old method : 0.038999080657958984 create new chi : 4.78583025932312 time to delete rle : 0.0021522045135498047 batch 1 Loaded 20 chid ids of type : 3726 Number RLEs to save : 15149 TO DO : save crop sub photo not yet done ! save time : 0.8397905826568604 nb_obj : 31 nb_hashtags : 7 time to prepare the origin masks : 10.54846453666687 time for calcul the mask position with numpy : 0.6115071773529053 nb_pixel_total : 5762190 time to create 1 rle with new method : 0.9892518520355225 time for calcul the mask position with numpy : 0.029041767120361328 nb_pixel_total : 1156 time to create 1 rle with old method : 0.0014777183532714844 time for calcul the mask position with numpy : 0.029559850692749023 nb_pixel_total : 32499 time to create 1 rle with old method : 0.037561893463134766 time for calcul the mask position with numpy : 0.030426502227783203 nb_pixel_total : 112667 time to create 1 rle with old method : 0.12952256202697754 time for calcul the mask position with numpy : 0.02907848358154297 nb_pixel_total : 22515 time to create 1 rle with old method : 0.026261568069458008 time for calcul the mask position with numpy : 0.02986931800842285 nb_pixel_total : 62673 time to create 1 rle with old method : 0.07340049743652344 time for calcul the mask position with numpy : 0.028432846069335938 nb_pixel_total : 15669 time to create 1 rle with old method : 0.023834228515625 time for calcul the mask position with numpy : 0.03528785705566406 nb_pixel_total : 3509 time to create 1 rle with old method : 0.0044519901275634766 time for calcul the mask position with numpy : 0.0340571403503418 nb_pixel_total : 206466 time to create 1 rle with new method : 0.5686566829681396 time for calcul the mask position with numpy : 0.02894282341003418 nb_pixel_total : 17082 time to create 1 rle with old method : 0.02158975601196289 time for calcul the mask position with numpy : 0.029630661010742188 nb_pixel_total : 23224 time to create 1 rle with old method : 0.027524471282958984 time for calcul the mask position with numpy : 0.029238224029541016 nb_pixel_total : 18172 time to create 1 rle with old method : 0.021504640579223633 time for calcul the mask position with numpy : 0.028468847274780273 nb_pixel_total : 2070 time to create 1 rle with old method : 0.002427816390991211 time for calcul the mask position with numpy : 0.029349327087402344 nb_pixel_total : 111902 time to create 1 rle with old method : 0.1387031078338623 time for calcul the mask position with numpy : 0.028873682022094727 nb_pixel_total : 17101 time to create 1 rle with old method : 0.019528627395629883 time for calcul the mask position with numpy : 0.02847886085510254 nb_pixel_total : 16254 time to create 1 rle with old method : 0.019208192825317383 time for calcul the mask position with numpy : 0.028961658477783203 nb_pixel_total : 47863 time to create 1 rle with old method : 0.055500030517578125 time for calcul the mask position with numpy : 0.0341339111328125 nb_pixel_total : 10577 time to create 1 rle with old method : 0.013213396072387695 time for calcul the mask position with numpy : 0.03225255012512207 nb_pixel_total : 164431 time to create 1 rle with new method : 0.44759392738342285 time for calcul the mask position with numpy : 0.028070688247680664 nb_pixel_total : 23431 time to create 1 rle with old method : 0.026409149169921875 time for calcul the mask position with numpy : 0.028150081634521484 nb_pixel_total : 31087 time to create 1 rle with old method : 0.036243438720703125 time for calcul the mask position with numpy : 0.027487754821777344 nb_pixel_total : 5308 time to create 1 rle with old method : 0.005833148956298828 time for calcul the mask position with numpy : 0.02729511260986328 nb_pixel_total : 8638 time to create 1 rle with old method : 0.009983539581298828 time for calcul the mask position with numpy : 0.027156352996826172 nb_pixel_total : 15283 time to create 1 rle with old method : 0.016869783401489258 time for calcul the mask position with numpy : 0.027870893478393555 nb_pixel_total : 23786 time to create 1 rle with old method : 0.02722024917602539 time for calcul the mask position with numpy : 0.028710126876831055 nb_pixel_total : 101069 time to create 1 rle with old method : 0.11117839813232422 time for calcul the mask position with numpy : 0.02862691879272461 nb_pixel_total : 111544 time to create 1 rle with old method : 0.12229299545288086 time for calcul the mask position with numpy : 0.028247833251953125 nb_pixel_total : 43527 time to create 1 rle with old method : 0.04753446578979492 time for calcul the mask position with numpy : 0.027741432189941406 nb_pixel_total : 5158 time to create 1 rle with old method : 0.005666017532348633 time for calcul the mask position with numpy : 0.02744150161743164 nb_pixel_total : 14096 time to create 1 rle with old method : 0.016457080841064453 time for calcul the mask position with numpy : 0.027382850646972656 nb_pixel_total : 16313 time to create 1 rle with old method : 0.018680572509765625 time for calcul the mask position with numpy : 0.027634143829345703 nb_pixel_total : 2980 time to create 1 rle with old method : 0.003504037857055664 create new chi : 4.669220924377441 time to delete rle : 0.0017058849334716797 batch 1 Loaded 32 chid ids of type : 3726 Number RLEs to save : 19138 TO DO : save crop sub photo not yet done ! save time : 1.1917905807495117 nb_obj : 20 nb_hashtags : 7 time to prepare the origin masks : 11.007362604141235 time for calcul the mask position with numpy : 0.5203201770782471 nb_pixel_total : 6182430 time to create 1 rle with new method : 0.6599135398864746 time for calcul the mask position with numpy : 0.03542733192443848 nb_pixel_total : 99320 time to create 1 rle with old method : 0.11268162727355957 time for calcul the mask position with numpy : 0.03380274772644043 nb_pixel_total : 93455 time to create 1 rle with old method : 0.10567736625671387 time for calcul the mask position with numpy : 0.03335237503051758 nb_pixel_total : 31869 time to create 1 rle with old method : 0.037369489669799805 time for calcul the mask position with numpy : 0.03399229049682617 nb_pixel_total : 24761 time to create 1 rle with old method : 0.029623746871948242 time for calcul the mask position with numpy : 0.03474998474121094 nb_pixel_total : 85127 time to create 1 rle with old method : 0.09688115119934082 time for calcul the mask position with numpy : 0.03633689880371094 nb_pixel_total : 69518 time to create 1 rle with old method : 0.07896637916564941 time for calcul the mask position with numpy : 0.037447214126586914 nb_pixel_total : 3936 time to create 1 rle with old method : 0.004790782928466797 time for calcul the mask position with numpy : 0.03687620162963867 nb_pixel_total : 4210 time to create 1 rle with old method : 0.004761934280395508 time for calcul the mask position with numpy : 0.03599905967712402 nb_pixel_total : 2945 time to create 1 rle with old method : 0.003497600555419922 time for calcul the mask position with numpy : 0.034096717834472656 nb_pixel_total : 1460 time to create 1 rle with old method : 0.0017642974853515625 time for calcul the mask position with numpy : 0.03915691375732422 nb_pixel_total : 131328 time to create 1 rle with old method : 0.1464850902557373 time for calcul the mask position with numpy : 0.034499406814575195 nb_pixel_total : 20842 time to create 1 rle with old method : 0.023331165313720703 time for calcul the mask position with numpy : 0.03544020652770996 nb_pixel_total : 1307 time to create 1 rle with old method : 0.00152587890625 time for calcul the mask position with numpy : 0.03407692909240723 nb_pixel_total : 42955 time to create 1 rle with old method : 0.05196118354797363 time for calcul the mask position with numpy : 0.03644204139709473 nb_pixel_total : 2890 time to create 1 rle with old method : 0.00337982177734375 time for calcul the mask position with numpy : 0.03453230857849121 nb_pixel_total : 7518 time to create 1 rle with old method : 0.008897066116333008 time for calcul the mask position with numpy : 0.03559708595275879 nb_pixel_total : 3652 time to create 1 rle with old method : 0.004166603088378906 time for calcul the mask position with numpy : 0.03645467758178711 nb_pixel_total : 9633 time to create 1 rle with old method : 0.010833740234375 time for calcul the mask position with numpy : 0.03666806221008301 nb_pixel_total : 202406 time to create 1 rle with new method : 0.6379680633544922 time for calcul the mask position with numpy : 0.031919002532958984 nb_pixel_total : 28678 time to create 1 rle with old method : 0.05045604705810547 create new chi : 3.3572561740875244 time to delete rle : 0.0019180774688720703 batch 1 Loaded 21 chid ids of type : 3726 Number RLEs to save : 12245 TO DO : save crop sub photo not yet done ! save time : 0.6990058422088623 nb_obj : 11 nb_hashtags : 6 time to prepare the origin masks : 8.894370317459106 time for calcul the mask position with numpy : 0.42787647247314453 nb_pixel_total : 5893132 time to create 1 rle with new method : 0.7579343318939209 time for calcul the mask position with numpy : 0.02252030372619629 nb_pixel_total : 167212 time to create 1 rle with new method : 0.5065739154815674 time for calcul the mask position with numpy : 0.02254462242126465 nb_pixel_total : 26972 time to create 1 rle with old method : 0.03094029426574707 time for calcul the mask position with numpy : 0.02193474769592285 nb_pixel_total : 21818 time to create 1 rle with old method : 0.03429388999938965 time for calcul the mask position with numpy : 0.024341344833374023 nb_pixel_total : 75212 time to create 1 rle with old method : 0.09311604499816895 time for calcul the mask position with numpy : 0.024853944778442383 nb_pixel_total : 149545 time to create 1 rle with old method : 0.1699364185333252 time for calcul the mask position with numpy : 0.023718833923339844 nb_pixel_total : 99751 time to create 1 rle with old method : 0.11697196960449219 time for calcul the mask position with numpy : 0.021109342575073242 nb_pixel_total : 61015 time to create 1 rle with old method : 0.06914806365966797 time for calcul the mask position with numpy : 0.020584821701049805 nb_pixel_total : 79372 time to create 1 rle with old method : 0.08890795707702637 time for calcul the mask position with numpy : 0.021550416946411133 nb_pixel_total : 63831 time to create 1 rle with old method : 0.07263326644897461 time for calcul the mask position with numpy : 0.02121281623840332 nb_pixel_total : 52992 time to create 1 rle with old method : 0.06266117095947266 time for calcul the mask position with numpy : 0.024548053741455078 nb_pixel_total : 359388 time to create 1 rle with new method : 0.7669553756713867 create new chi : 3.5289251804351807 time to delete rle : 0.0018036365509033203 batch 1 Loaded 12 chid ids of type : 3726 Number RLEs to save : 13591 TO DO : save crop sub photo not yet done ! save time : 0.8027245998382568 nb_obj : 14 nb_hashtags : 5 time to prepare the origin masks : 7.878848075866699 time for calcul the mask position with numpy : 0.532602071762085 nb_pixel_total : 6477325 time to create 1 rle with new method : 0.9328505992889404 time for calcul the mask position with numpy : 0.030568838119506836 nb_pixel_total : 38467 time to create 1 rle with old method : 0.04223060607910156 time for calcul the mask position with numpy : 0.0211942195892334 nb_pixel_total : 78837 time to create 1 rle with old method : 0.08313822746276855 time for calcul the mask position with numpy : 0.020079612731933594 nb_pixel_total : 72524 time to create 1 rle with old method : 0.07930231094360352 time for calcul the mask position with numpy : 0.019397974014282227 nb_pixel_total : 1005 time to create 1 rle with old method : 0.0010967254638671875 time for calcul the mask position with numpy : 0.01981496810913086 nb_pixel_total : 20555 time to create 1 rle with old method : 0.021370410919189453 time for calcul the mask position with numpy : 0.0203855037689209 nb_pixel_total : 4073 time to create 1 rle with old method : 0.0044324398040771484 time for calcul the mask position with numpy : 0.02004408836364746 nb_pixel_total : 49598 time to create 1 rle with old method : 0.05580019950866699 time for calcul the mask position with numpy : 0.020093202590942383 nb_pixel_total : 12963 time to create 1 rle with old method : 0.014665365219116211 time for calcul the mask position with numpy : 0.020922183990478516 nb_pixel_total : 5095 time to create 1 rle with old method : 0.005479574203491211 time for calcul the mask position with numpy : 0.019913673400878906 nb_pixel_total : 102254 time to create 1 rle with old method : 0.11166143417358398 time for calcul the mask position with numpy : 0.021097660064697266 nb_pixel_total : 41918 time to create 1 rle with old method : 0.04944419860839844 time for calcul the mask position with numpy : 0.023273944854736328 nb_pixel_total : 60677 time to create 1 rle with old method : 0.09073591232299805 time for calcul the mask position with numpy : 0.03186202049255371 nb_pixel_total : 70517 time to create 1 rle with old method : 0.09421801567077637 time for calcul the mask position with numpy : 0.029465198516845703 nb_pixel_total : 14432 time to create 1 rle with old method : 0.017521381378173828 create new chi : 2.489065170288086 time to delete rle : 0.0011339187622070312 batch 1 Loaded 15 chid ids of type : 3726 Number RLEs to save : 10854 TO DO : save crop sub photo not yet done ! save time : 0.6427650451660156 nb_obj : 20 nb_hashtags : 7 time to prepare the origin masks : 14.093180179595947 time for calcul the mask position with numpy : 0.5321841239929199 nb_pixel_total : 6270953 time to create 1 rle with new method : 0.574721097946167 time for calcul the mask position with numpy : 0.023070812225341797 nb_pixel_total : 90481 time to create 1 rle with old method : 0.12304449081420898 time for calcul the mask position with numpy : 0.021574735641479492 nb_pixel_total : 37016 time to create 1 rle with old method : 0.044481515884399414 time for calcul the mask position with numpy : 0.020170211791992188 nb_pixel_total : 6974 time to create 1 rle with old method : 0.007994651794433594 time for calcul the mask position with numpy : 0.02220773696899414 nb_pixel_total : 5922 time to create 1 rle with old method : 0.007103443145751953 time for calcul the mask position with numpy : 0.023621082305908203 nb_pixel_total : 23917 time to create 1 rle with old method : 0.02854466438293457 time for calcul the mask position with numpy : 0.021184444427490234 nb_pixel_total : 26414 time to create 1 rle with old method : 0.02971196174621582 time for calcul the mask position with numpy : 0.0210723876953125 nb_pixel_total : 2313 time to create 1 rle with old method : 0.0027332305908203125 time for calcul the mask position with numpy : 0.021515846252441406 nb_pixel_total : 46647 time to create 1 rle with old method : 0.05295705795288086 time for calcul the mask position with numpy : 0.02344512939453125 nb_pixel_total : 10557 time to create 1 rle with old method : 0.012511730194091797 time for calcul the mask position with numpy : 0.02285003662109375 nb_pixel_total : 44312 time to create 1 rle with old method : 0.05033278465270996 time for calcul the mask position with numpy : 0.02154064178466797 nb_pixel_total : 107854 time to create 1 rle with old method : 0.12053442001342773 time for calcul the mask position with numpy : 0.021785974502563477 nb_pixel_total : 1488 time to create 1 rle with old method : 0.001993417739868164 time for calcul the mask position with numpy : 0.022002696990966797 nb_pixel_total : 34696 time to create 1 rle with old method : 0.03841137886047363 time for calcul the mask position with numpy : 0.022700786590576172 nb_pixel_total : 155629 time to create 1 rle with new method : 0.4423189163208008 time for calcul the mask position with numpy : 0.021484375 nb_pixel_total : 15177 time to create 1 rle with old method : 0.018024206161499023 time for calcul the mask position with numpy : 0.021923065185546875 nb_pixel_total : 46603 time to create 1 rle with old method : 0.051659584045410156 time for calcul the mask position with numpy : 0.021587848663330078 nb_pixel_total : 49991 time to create 1 rle with old method : 0.056185007095336914 time for calcul the mask position with numpy : 0.024097442626953125 nb_pixel_total : 18779 time to create 1 rle with old method : 0.021301746368408203 time for calcul the mask position with numpy : 0.0216219425201416 nb_pixel_total : 4502 time to create 1 rle with old method : 0.0051763057708740234 time for calcul the mask position with numpy : 0.02235555648803711 nb_pixel_total : 50015 time to create 1 rle with old method : 0.056725263595581055 create new chi : 2.7870161533355713 time to delete rle : 0.001323699951171875 batch 1 Loaded 21 chid ids of type : 3726 Number RLEs to save : 14498 TO DO : save crop sub photo not yet done ! save time : 0.9776487350463867 nb_obj : 19 nb_hashtags : 7 time to prepare the origin masks : 13.192663192749023 time for calcul the mask position with numpy : 0.8493940830230713 nb_pixel_total : 6298235 time to create 1 rle with new method : 1.2713217735290527 time for calcul the mask position with numpy : 0.03506612777709961 nb_pixel_total : 19309 time to create 1 rle with old method : 0.024864912033081055 time for calcul the mask position with numpy : 0.03636312484741211 nb_pixel_total : 15690 time to create 1 rle with old method : 0.018832683563232422 time for calcul the mask position with numpy : 0.035237789154052734 nb_pixel_total : 28743 time to create 1 rle with old method : 0.03254532814025879 time for calcul the mask position with numpy : 0.034836530685424805 nb_pixel_total : 2569 time to create 1 rle with old method : 0.003010988235473633 time for calcul the mask position with numpy : 0.03569626808166504 nb_pixel_total : 2857 time to create 1 rle with old method : 0.003312826156616211 time for calcul the mask position with numpy : 0.03469085693359375 nb_pixel_total : 4026 time to create 1 rle with old method : 0.004824161529541016 time for calcul the mask position with numpy : 0.0351560115814209 nb_pixel_total : 22616 time to create 1 rle with old method : 0.025850296020507812 time for calcul the mask position with numpy : 0.03599119186401367 nb_pixel_total : 48332 time to create 1 rle with old method : 0.05393075942993164 time for calcul the mask position with numpy : 0.03351712226867676 nb_pixel_total : 62884 time to create 1 rle with old method : 0.06832671165466309 time for calcul the mask position with numpy : 0.03417158126831055 nb_pixel_total : 32499 time to create 1 rle with old method : 0.036513328552246094 time for calcul the mask position with numpy : 0.03320789337158203 nb_pixel_total : 15529 time to create 1 rle with old method : 0.016979217529296875 time for calcul the mask position with numpy : 0.03368973731994629 nb_pixel_total : 40575 time to create 1 rle with old method : 0.04655313491821289 time for calcul the mask position with numpy : 0.04574847221374512 nb_pixel_total : 256809 time to create 1 rle with new method : 0.5854892730712891 time for calcul the mask position with numpy : 0.0346379280090332 nb_pixel_total : 82993 time to create 1 rle with old method : 0.11820387840270996 time for calcul the mask position with numpy : 0.03138256072998047 nb_pixel_total : 1219 time to create 1 rle with old method : 0.0016355514526367188 time for calcul the mask position with numpy : 0.021368980407714844 nb_pixel_total : 26467 time to create 1 rle with old method : 0.029882192611694336 time for calcul the mask position with numpy : 0.02135491371154785 nb_pixel_total : 33017 time to create 1 rle with old method : 0.03736305236816406 time for calcul the mask position with numpy : 0.0224306583404541 nb_pixel_total : 47597 time to create 1 rle with old method : 0.05421757698059082 time for calcul the mask position with numpy : 0.02445507049560547 nb_pixel_total : 8274 time to create 1 rle with old method : 0.009417295455932617 create new chi : 3.9731719493865967 time to delete rle : 0.0017707347869873047 batch 1 Loaded 20 chid ids of type : 3726 Number RLEs to save : 12275 TO DO : save crop sub photo not yet done ! save time : 0.7195453643798828 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 4.423732280731201 time for calcul the mask position with numpy : 0.5783190727233887 nb_pixel_total : 3757968 time to create 1 rle with new method : 1.5556800365447998 time for calcul the mask position with numpy : 0.023769378662109375 nb_pixel_total : 68526 time to create 1 rle with old method : 0.08273649215698242 time for calcul the mask position with numpy : 0.8185162544250488 nb_pixel_total : 3223746 time to create 1 rle with new method : 0.9976987838745117 create new chi : 4.122361421585083 time to delete rle : 0.0010178089141845703 batch 1 Loaded 3 chid ids of type : 3726 Number RLEs to save : 8074 TO DO : save crop sub photo not yet done ! save time : 1.3487794399261475 map_output_result : {1339879029: (0.0, 'Should be the crop_list due to order', 0), 1339879005: (0.0, 'Should be the crop_list due to order', 0), 1339878979: (0.0, 'Should be the crop_list due to order', 0), 1339878976: (0.0, 'Should be the crop_list due to order', 0), 1339878973: (0.0, 'Should be the crop_list due to order', 0), 1339878960: (0.0, 'Should be the crop_list due to order', 0), 1339878953: (0.0, 'Should be the crop_list due to order', 0), 1339878950: (0.0, 'Should be the crop_list due to order', 0), 1339878670: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1339879029, 1339879005, 1339878979, 1339878976, 1339878973, 1339878960, 1339878953, 1339878950, 1339878670] Looping around the photos to save general results len do output : 9 /1339879029.Didn't retrieve data . /1339879005.Didn't retrieve data . /1339878979.Didn't retrieve data . /1339878976.Didn't retrieve data . /1339878973.Didn't retrieve data . /1339878960.Didn't retrieve data . /1339878953.Didn't retrieve data . /1339878950.Didn't retrieve data . /1339878670.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339879029', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339879005', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878979', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878976', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878973', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878960', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878953', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878950', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878670', None, None, None, None, None, '2619516') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 27 time used for this insertion : 0.013001203536987305 save_final save missing photos in datou_result : time spend for datou_step_exec : 139.21850991249084 time spend to save output : 0.013504743576049805 total time spend for step 5 : 139.2320146560669 step6:crop_condition Thu Feb 27 09:18:47 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 9 ! batch 1 Loaded 161 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! About to upload 7 photos upload in portfolio : 4869462 init cache_photo without model_param we have 7 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740644332_2861688 we have uploaded 7 photos in the portfolio 4869462 time of upload the photos Elapsed time : 9.47945785522461 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740644343_2861688 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.0066733360290527 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 4869462 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740644347_2861688 we have uploaded 6 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.8361663818359375 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740644351_2861688 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.0240404605865479 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! About to upload 7 photos upload in portfolio : 4869462 init cache_photo without model_param we have 7 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740644355_2861688 we have uploaded 7 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.9208967685699463 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740644361_2861688 we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.5212349891662598 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740644364_2861688 we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.1716744899749756 we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740644367_2861688 we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.50852632522583 we have finished the crop for the class : metal delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1339879029, 1339879005, 1339878979, 1339878976, 1339878973, 1339878960, 1339878953, 1339878950, 1339878670] Looping around the photos to save general results len do output : 32 /1339884799Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884800Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884801Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884802Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884803Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884805Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884806Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884809Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884810Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884811Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884814Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884815Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884816Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884817Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884818Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884819Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884820Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884821Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884822Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884824Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884825Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884826Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884827Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884828Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884829Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884830Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884832Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884833Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884834Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884835Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884836Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339884837Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339879029', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339879005', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878979', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878976', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878973', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878960', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878953', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878950', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878670', None, None, None, None, None, '2619516') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 105 time used for this insertion : 0.10987091064453125 save_final save missing photos in datou_result : time spend for datou_step_exec : 41.52090406417847 time spend to save output : 0.11084818840026855 total time spend for step 6 : 41.631752252578735 step7:ventilate_hashtags_in_portfolio Thu Feb 27 09:19:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 20879433 get user id for portfolio 20879433 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20879433 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Teint_Dans_La_Masse','cartonnette','plastique','flou','Carton_brun','kraft','autre_refus','mal_croppe','metal','environnement','Carton_gris','papier')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20879433 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Teint_Dans_La_Masse','cartonnette','plastique','flou','Carton_brun','kraft','autre_refus','mal_croppe','metal','environnement','Carton_gris','papier')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20879433 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Teint_Dans_La_Masse','cartonnette','plastique','flou','Carton_brun','kraft','autre_refus','mal_croppe','metal','environnement','Carton_gris','papier')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/20879519,20879520,20879521,20879522,20879523,20879524,20879525,20879526,20879527,20879528,20879529,20879530?tags=Teint_Dans_La_Masse,cartonnette,plastique,flou,Carton_brun,kraft,autre_refus,mal_croppe,metal,environnement,Carton_gris,papier Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1339879029, 1339879005, 1339878979, 1339878976, 1339878973, 1339878960, 1339878953, 1339878950, 1339878670] Looping around the photos to save general results len do output : 1 /20879433. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339879029', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339879005', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878979', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878976', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878973', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878960', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878953', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878950', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878670', None, None, None, None, None, '2619516') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.01749253273010254 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.746840476989746 time spend to save output : 0.017725706100463867 total time spend for step 7 : 1.76456618309021 step8:final Thu Feb 27 09:19:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1339879029: ('0.08366272061120135',), 1339879005: ('0.08366272061120135',), 1339878979: ('0.08366272061120135',), 1339878976: ('0.08366272061120135',), 1339878973: ('0.08366272061120135',), 1339878960: ('0.08366272061120135',), 1339878953: ('0.08366272061120135',), 1339878950: ('0.08366272061120135',), 1339878670: ('0.08366272061120135',)} new output for save of step final : {1339879029: ('0.08366272061120135',), 1339879005: ('0.08366272061120135',), 1339878979: ('0.08366272061120135',), 1339878976: ('0.08366272061120135',), 1339878973: ('0.08366272061120135',), 1339878960: ('0.08366272061120135',), 1339878953: ('0.08366272061120135',), 1339878950: ('0.08366272061120135',), 1339878670: ('0.08366272061120135',)} [1339879029, 1339879005, 1339878979, 1339878976, 1339878973, 1339878960, 1339878953, 1339878950, 1339878670] Looping around the photos to save general results len do output : 9 /1339879029.Didn't retrieve data . /1339879005.Didn't retrieve data . /1339878979.Didn't retrieve data . /1339878976.Didn't retrieve data . /1339878973.Didn't retrieve data . /1339878960.Didn't retrieve data . /1339878953.Didn't retrieve data . /1339878950.Didn't retrieve data . /1339878670.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339879029', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339879005', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878979', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878976', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878973', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878960', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878953', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878950', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878670', None, None, None, None, None, '2619516') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 27 time used for this insertion : 0.014661788940429688 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.16974353790283203 time spend to save output : 0.015129327774047852 total time spend for step 8 : 0.18487286567687988 step9:velours_tree Thu Feb 27 09:19:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.04037737846374512 time spend to save output : 2.956390380859375e-05 total time spend for step 9 : 0.04040694236755371 step10:send_mail_cod Thu Feb 27 09:19:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P20879433_27-02-2025_09_19_31.pdf 20879520 change filename to text .change filename to text .change filename to text .imagette208795201740644371 20879521 change filename to text .change filename to text .change filename to text .change filename to text .imagette208795211740644371 20879522 imagette208795221740644371 20879524 change filename to text .imagette208795241740644371 20879525 change filename to text .change filename to text .change filename to text .imagette208795251740644371 20879526 imagette208795261740644372 20879527 change filename to text .imagette208795271740644372 20879519 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette208795191740644372 20879523 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette208795231740644372 20879529 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette208795291740644373 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20879433 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20879519,20879520,20879521,20879522,20879523,20879524,20879525,20879526,20879527,20879528,20879529,20879530?tags=Teint_Dans_La_Masse,cartonnette,plastique,flou,Carton_brun,kraft,autre_refus,mal_croppe,metal,environnement,Carton_gris,papier your option no_mail is active, we will not send the real mail to your client args[1339879029] : ('0.08366272061120135',) no score found for photo 1339879029 We are sending mail with results at report@fotonower.com args[1339879005] : ('0.08366272061120135',) no score found for photo 1339879005 We are sending mail with results at report@fotonower.com args[1339878979] : ('0.08366272061120135',) no score found for photo 1339878979 We are sending mail with results at report@fotonower.com args[1339878976] : ('0.08366272061120135',) no score found for photo 1339878976 We are sending mail with results at report@fotonower.com args[1339878973] : ('0.08366272061120135',) no score found for photo 1339878973 We are sending mail with results at report@fotonower.com args[1339878960] : ('0.08366272061120135',) no score found for photo 1339878960 We are sending mail with results at report@fotonower.com args[1339878953] : ('0.08366272061120135',) no score found for photo 1339878953 We are sending mail with results at report@fotonower.com args[1339878950] : ('0.08366272061120135',) no score found for photo 1339878950 We are sending mail with results at report@fotonower.com args[1339878670] : ('0.08366272061120135',) no score found for photo 1339878670 We are sending mail with results at report@fotonower.com refus_total : 0.08366272061120135 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20879433 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1339878605,1339878608,1339878612,1339878950,1339878953,1339878960,1339878976,1339878979,1339878603,1339878670,1339878973,1339879005,1339879029) Found this number of photos: 13 begin to download photo : 1339878605 begin to download photo : 1339878950 begin to download photo : 1339878976 begin to download photo : 1339878670 begin to download photo : 1339879029 download finish for photo 1339878670 begin to download photo : 1339878973 download finish for photo 1339879029 download finish for photo 1339878605 begin to download photo : 1339878608 download finish for photo 1339878950 begin to download photo : 1339878953 download finish for photo 1339878976 begin to download photo : 1339878979 download finish for photo 1339878973 begin to download photo : 1339879005 download finish for photo 1339878608 begin to download photo : 1339878612 download finish for photo 1339878953 begin to download photo : 1339878960 download finish for photo 1339878979 begin to download photo : 1339878603 download finish for photo 1339878612 download finish for photo 1339878960 download finish for photo 1339879005 download finish for photo 1339878603 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20879433_27-02-2025_09_19_31.pdf results_Auto_P20879433_27-02-2025_09_19_31.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20879433_27-02-2025_09_19_31.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3459','20879433','results_Auto_P20879433_27-02-2025_09_19_31.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20879433_27-02-2025_09_19_31.pdf','pdf','','0.62','0.08366272061120135') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1339879029, 1339879005, 1339878979, 1339878976, 1339878973, 1339878960, 1339878953, 1339878950, 1339878670] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339879029', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339879005', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878979', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878976', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878973', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878960', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878953', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878950', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878670', None, None, None, None, None, '2619516') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.012386083602905273 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.906128406524658 time spend to save output : 0.012691497802734375 total time spend for step 10 : 4.918819904327393 step11:split_time_score Thu Feb 27 09:19:36 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('08', 13),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 27022025 20879433 Nombre de photos uploadées : 13 / 23040 (0%) 27022025 20879433 Nombre de photos taguées (types de déchets): 0 / 13 (0%) 27022025 20879433 Nombre de photos taguées (volume) : 0 / 13 (0%) elapsed_time : load_data_split_time_score 2.1457672119140625e-06 elapsed_time : order_list_meta_photo_and_scores 6.9141387939453125e-06 ????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0007088184356689453 elapsed_time : insert_dashboard_record_day_entry 0.02219080924987793 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.05676013498732907 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20879430_27-02-2025_09_17_36.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20879430 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20879430 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20879431 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20879432 order by id desc limit 1 Qualite : 0.08366272061120135 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20879433_27-02-2025_09_19_31.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20879433 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20879433 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20879434 order by id desc limit 1 NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'27022025': {'nb_upload': 13, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1339879029, 1339879005, 1339878979, 1339878976, 1339878973, 1339878960, 1339878953, 1339878950, 1339878670] Looping around the photos to save general results len do output : 1 /20879433Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339879029', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339879005', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878979', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878976', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878973', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878960', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878953', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878950', None, None, None, None, None, '2619516') ('3459', None, None, None, None, None, None, None, '2619516') ('3459', '20879433', '1339878670', None, None, None, None, None, '2619516') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.013406515121459961 save_final save missing photos in datou_result : begin to insert list_values into mtr_datou_result : length of list_values in save_final : 1 time used for this insertion : 0.010718107223510742 time spend for datou_step_exec : 2.657888412475586 time spend to save output : 0.024418115615844727 total time spend for step 11 : 2.6823065280914307 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 152.08user 135.01system 7:14.41elapsed 66%CPU (0avgtext+0avgdata 4633928maxresident)k 64056inputs+112976outputs (1319major+13171398minor)pagefaults 0swaps