python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 4234 -l 5 ' -s datou_current_4234 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 887412 load datou : 4234 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load THCL from format json or kwargs add thcl : 3237 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig add pdt : 5561 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4234, datou_cur_ids : ['2538723'] with mtr_portfolio_ids : ['20162692'] and first list_photo_ids : [] new path : /proc/887412/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, brightness, blur_detection, rle_unique_nms_with_priority, crop_condition, thcl, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , BFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 3 ; length of list_pids : 3 ; length of list_args : 3 time to download the photos : 0.4073200225830078 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Mon Feb 3 01:20:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10774 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-03 01:20:30.936061: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-03 01:20:30.944147: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493020000 Hz 2025-02-03 01:20:30.945686: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f317c000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-03 01:20:30.945738: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-03 01:20:30.948283: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-03 01:20:31.164834: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x40f84460 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-03 01:20:31.164890: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-03 01:20:31.166405: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-03 01:20:31.166807: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-03 01:20:31.169979: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-03 01:20:31.173050: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-03 01:20:31.173555: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-03 01:20:31.176919: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-03 01:20:31.177923: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-03 01:20:31.182481: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-03 01:20:31.183976: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-03 01:20:31.184060: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-03 01:20:31.184825: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-03 01:20:31.184841: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-03 01:20:31.184850: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-03 01:20:31.186118: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9985 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-03 01:20:31.434381: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-03 01:20:31.434523: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-03 01:20:31.434554: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-03 01:20:31.434584: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-03 01:20:31.434611: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-03 01:20:31.434639: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-03 01:20:31.434665: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-03 01:20:31.434693: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-03 01:20:31.436811: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-03 01:20:31.437923: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-03 01:20:31.437951: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-03 01:20:31.437965: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-03 01:20:31.437977: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-03 01:20:31.437990: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-03 01:20:31.438002: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-03 01:20:31.438014: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-03 01:20:31.438026: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-03 01:20:31.439154: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-03 01:20:31.439186: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-03 01:20:31.439194: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-03 01:20:31.439201: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-03 01:20:31.440439: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9985 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-03 01:20:39.272363: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-03 01:20:39.428142: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 3 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -118.34453 max: 149.46328 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 7.534027099609375e-05 nb_pixel_total : 816 time to create 1 rle with old method : 0.0015904903411865234 length of segment : 35 time for calcul the mask position with numpy : 5.6743621826171875e-05 nb_pixel_total : 901 time to create 1 rle with old method : 0.0016636848449707031 length of segment : 37 time for calcul the mask position with numpy : 4.482269287109375e-05 nb_pixel_total : 370 time to create 1 rle with old method : 0.0006642341613769531 length of segment : 32 time for calcul the mask position with numpy : 7.033348083496094e-05 nb_pixel_total : 2136 time to create 1 rle with old method : 0.0037207603454589844 length of segment : 32 time for calcul the mask position with numpy : 5.745887756347656e-05 nb_pixel_total : 880 time to create 1 rle with old method : 0.0016303062438964844 length of segment : 51 time for calcul the mask position with numpy : 5.269050598144531e-05 nb_pixel_total : 858 time to create 1 rle with old method : 0.0015485286712646484 length of segment : 51 time for calcul the mask position with numpy : 6.532669067382812e-05 nb_pixel_total : 1983 time to create 1 rle with old method : 0.0035552978515625 length of segment : 30 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.87969 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 7.176399230957031e-05 nb_pixel_total : 2372 time to create 1 rle with old method : 0.00292205810546875 length of segment : 58 time for calcul the mask position with numpy : 9.846687316894531e-05 nb_pixel_total : 3781 time to create 1 rle with old method : 0.004964113235473633 length of segment : 60 time for calcul the mask position with numpy : 3.4332275390625e-05 nb_pixel_total : 126 time to create 1 rle with old method : 0.00019788742065429688 length of segment : 12 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 871 time to create 1 rle with old method : 0.0010721683502197266 length of segment : 38 time for calcul the mask position with numpy : 4.1484832763671875e-05 nb_pixel_total : 757 time to create 1 rle with old method : 0.000934600830078125 length of segment : 37 time for calcul the mask position with numpy : 3.647804260253906e-05 nb_pixel_total : 226 time to create 1 rle with old method : 0.0003333091735839844 length of segment : 51 Processing 1 images image shape: (400, 400, 3) min: 28.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -89.87969 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 0.00011348724365234375 nb_pixel_total : 6333 time to create 1 rle with old method : 0.00730586051940918 length of segment : 102 time for calcul the mask position with numpy : 9.870529174804688e-05 nb_pixel_total : 4842 time to create 1 rle with old method : 0.005736351013183594 length of segment : 53 time for calcul the mask position with numpy : 4.7206878662109375e-05 nb_pixel_total : 1011 time to create 1 rle with old method : 0.0012226104736328125 length of segment : 43 time for calcul the mask position with numpy : 5.125999450683594e-05 nb_pixel_total : 1507 time to create 1 rle with old method : 0.0018911361694335938 length of segment : 30 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -110.59844 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 7.653236389160156e-05 nb_pixel_total : 530 time to create 1 rle with old method : 0.0014522075653076172 length of segment : 29 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 264 time to create 1 rle with old method : 0.0004589557647705078 length of segment : 23 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 146.38125 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 3.933906555175781e-05 nb_pixel_total : 323 time to create 1 rle with old method : 0.00044608116149902344 length of segment : 22 time for calcul the mask position with numpy : 3.4332275390625e-05 nb_pixel_total : 400 time to create 1 rle with old method : 0.000568389892578125 length of segment : 15 Processing 1 images image shape: (400, 400, 3) min: 40.00000 max: 217.00000 molded_images shape: (1, 640, 640, 3) min: -70.43047 max: 107.42031 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0013022422790527344 nb_pixel_total : 152768 time to create 1 rle with new method : 0.0019443035125732422 length of segment : 404 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -115.50078 max: 149.31484 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 4.220008850097656e-05 nb_pixel_total : 421 time to create 1 rle with old method : 0.0006132125854492188 length of segment : 20 time for calcul the mask position with numpy : 0.0001723766326904297 nb_pixel_total : 10434 time to create 1 rle with old method : 0.011766195297241211 length of segment : 129 time for calcul the mask position with numpy : 3.4809112548828125e-05 nb_pixel_total : 157 time to create 1 rle with old method : 0.0002334117889404297 length of segment : 18 time for calcul the mask position with numpy : 5.030632019042969e-05 nb_pixel_total : 1656 time to create 1 rle with old method : 0.001924276351928711 length of segment : 48 time for calcul the mask position with numpy : 3.24249267578125e-05 nb_pixel_total : 217 time to create 1 rle with old method : 0.00030732154846191406 length of segment : 18 time for calcul the mask position with numpy : 4.00543212890625e-05 nb_pixel_total : 722 time to create 1 rle with old method : 0.0009758472442626953 length of segment : 34 time for calcul the mask position with numpy : 5.459785461425781e-05 nb_pixel_total : 1446 time to create 1 rle with old method : 0.0018863677978515625 length of segment : 34 time for calcul the mask position with numpy : 3.147125244140625e-05 nb_pixel_total : 157 time to create 1 rle with old method : 0.0002384185791015625 length of segment : 13 time for calcul the mask position with numpy : 0.00014972686767578125 nb_pixel_total : 9832 time to create 1 rle with old method : 0.011092424392700195 length of segment : 122 Processing 1 images image shape: (400, 400, 3) min: 2.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -115.77813 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 4.2438507080078125e-05 nb_pixel_total : 448 time to create 1 rle with old method : 0.0005846023559570312 length of segment : 27 time for calcul the mask position with numpy : 3.337860107421875e-05 nb_pixel_total : 233 time to create 1 rle with old method : 0.00030875205993652344 length of segment : 21 time for calcul the mask position with numpy : 0.00028133392333984375 nb_pixel_total : 12644 time to create 1 rle with old method : 0.014177799224853516 length of segment : 112 time for calcul the mask position with numpy : 4.76837158203125e-05 nb_pixel_total : 808 time to create 1 rle with old method : 0.0010313987731933594 length of segment : 51 time for calcul the mask position with numpy : 3.552436828613281e-05 nb_pixel_total : 383 time to create 1 rle with old method : 0.0004990100860595703 length of segment : 24 time for calcul the mask position with numpy : 3.24249267578125e-05 nb_pixel_total : 203 time to create 1 rle with old method : 0.0003116130828857422 length of segment : 20 time for calcul the mask position with numpy : 3.8623809814453125e-05 nb_pixel_total : 667 time to create 1 rle with old method : 0.0008182525634765625 length of segment : 30 time for calcul the mask position with numpy : 0.00019025802612304688 nb_pixel_total : 12384 time to create 1 rle with old method : 0.014047861099243164 length of segment : 107 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -111.56328 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 9.918212890625e-05 nb_pixel_total : 2603 time to create 1 rle with old method : 0.0030782222747802734 length of segment : 158 time for calcul the mask position with numpy : 4.649162292480469e-05 nb_pixel_total : 585 time to create 1 rle with old method : 0.0008428096771240234 length of segment : 18 time for calcul the mask position with numpy : 4.6253204345703125e-05 nb_pixel_total : 1262 time to create 1 rle with old method : 0.0014939308166503906 length of segment : 37 time for calcul the mask position with numpy : 8.749961853027344e-05 nb_pixel_total : 1690 time to create 1 rle with old method : 0.0020923614501953125 length of segment : 118 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 807 time to create 1 rle with old method : 0.000982046127319336 length of segment : 40 time for calcul the mask position with numpy : 4.935264587402344e-05 nb_pixel_total : 1329 time to create 1 rle with old method : 0.0015873908996582031 length of segment : 39 time for calcul the mask position with numpy : 5.9604644775390625e-05 nb_pixel_total : 1228 time to create 1 rle with old method : 0.0016694068908691406 length of segment : 44 time for calcul the mask position with numpy : 9.584426879882812e-05 nb_pixel_total : 1933 time to create 1 rle with old method : 0.0023224353790283203 length of segment : 138 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 247.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 128.49062 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 12 time for calcul the mask position with numpy : 4.935264587402344e-05 nb_pixel_total : 821 time to create 1 rle with old method : 0.0010263919830322266 length of segment : 43 time for calcul the mask position with numpy : 3.814697265625e-05 nb_pixel_total : 508 time to create 1 rle with old method : 0.0006234645843505859 length of segment : 39 time for calcul the mask position with numpy : 3.409385681152344e-05 nb_pixel_total : 293 time to create 1 rle with old method : 0.0003895759582519531 length of segment : 33 time for calcul the mask position with numpy : 0.00017380714416503906 nb_pixel_total : 4973 time to create 1 rle with old method : 0.0061168670654296875 length of segment : 232 time for calcul the mask position with numpy : 5.221366882324219e-05 nb_pixel_total : 676 time to create 1 rle with old method : 0.0008616447448730469 length of segment : 65 time for calcul the mask position with numpy : 3.838539123535156e-05 nb_pixel_total : 541 time to create 1 rle with old method : 0.0006940364837646484 length of segment : 39 time for calcul the mask position with numpy : 6.151199340820312e-05 nb_pixel_total : 1620 time to create 1 rle with old method : 0.0019147396087646484 length of segment : 79 time for calcul the mask position with numpy : 7.224082946777344e-05 nb_pixel_total : 1533 time to create 1 rle with old method : 0.0018985271453857422 length of segment : 124 time for calcul the mask position with numpy : 0.0001876354217529297 nb_pixel_total : 4112 time to create 1 rle with old method : 0.0047054290771484375 length of segment : 196 time for calcul the mask position with numpy : 4.553794860839844e-05 nb_pixel_total : 721 time to create 1 rle with old method : 0.0008099079132080078 length of segment : 41 time for calcul the mask position with numpy : 5.435943603515625e-05 nb_pixel_total : 1534 time to create 1 rle with old method : 0.0018014907836914062 length of segment : 72 time for calcul the mask position with numpy : 4.029273986816406e-05 nb_pixel_total : 718 time to create 1 rle with old method : 0.0008807182312011719 length of segment : 41 Processing 1 images image shape: (280, 400, 3) min: 44.00000 max: 214.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 98.79531 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0009343624114990234 nb_pixel_total : 104543 time to create 1 rle with old method : 0.11431026458740234 length of segment : 282 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 147.26406 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 4.76837158203125e-05 nb_pixel_total : 893 time to create 1 rle with old method : 0.0012984275817871094 length of segment : 17 time for calcul the mask position with numpy : 2.8133392333984375e-05 nb_pixel_total : 65 time to create 1 rle with old method : 9.846687316894531e-05 length of segment : 11 time for calcul the mask position with numpy : 3.552436828613281e-05 nb_pixel_total : 318 time to create 1 rle with old method : 0.0004334449768066406 length of segment : 41 time for calcul the mask position with numpy : 3.2901763916015625e-05 nb_pixel_total : 472 time to create 1 rle with old method : 0.0005824565887451172 length of segment : 33 time for calcul the mask position with numpy : 4.57763671875e-05 nb_pixel_total : 1407 time to create 1 rle with old method : 0.001653909683227539 length of segment : 53 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 141.03359 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 3.7670135498046875e-05 nb_pixel_total : 70 time to create 1 rle with old method : 0.00012350082397460938 length of segment : 9 time for calcul the mask position with numpy : 5.555152893066406e-05 nb_pixel_total : 2151 time to create 1 rle with old method : 0.0025153160095214844 length of segment : 48 time for calcul the mask position with numpy : 3.8623809814453125e-05 nb_pixel_total : 368 time to create 1 rle with old method : 0.0005171298980712891 length of segment : 40 time for calcul the mask position with numpy : 7.104873657226562e-05 nb_pixel_total : 3360 time to create 1 rle with old method : 0.003787517547607422 length of segment : 98 time for calcul the mask position with numpy : 4.601478576660156e-05 nb_pixel_total : 901 time to create 1 rle with old method : 0.0010867118835449219 length of segment : 46 time for calcul the mask position with numpy : 3.838539123535156e-05 nb_pixel_total : 978 time to create 1 rle with old method : 0.0010809898376464844 length of segment : 47 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 149.61953 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 3.695487976074219e-05 nb_pixel_total : 74 time to create 1 rle with old method : 0.00015473365783691406 length of segment : 11 time for calcul the mask position with numpy : 3.1948089599609375e-05 nb_pixel_total : 164 time to create 1 rle with old method : 0.00022649765014648438 length of segment : 21 time for calcul the mask position with numpy : 2.9802322387695312e-05 nb_pixel_total : 273 time to create 1 rle with old method : 0.0003466606140136719 length of segment : 17 time for calcul the mask position with numpy : 2.7894973754882812e-05 nb_pixel_total : 142 time to create 1 rle with old method : 0.00017881393432617188 length of segment : 28 time for calcul the mask position with numpy : 2.9087066650390625e-05 nb_pixel_total : 28 time to create 1 rle with old method : 7.152557373046875e-05 length of segment : 5 time for calcul the mask position with numpy : 2.86102294921875e-05 nb_pixel_total : 35 time to create 1 rle with old method : 8.440017700195312e-05 length of segment : 6 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 141.97500 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 5.6743621826171875e-05 nb_pixel_total : 1191 time to create 1 rle with old method : 0.001483917236328125 length of segment : 128 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -115.59453 max: 149.90859 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 5.984306335449219e-05 nb_pixel_total : 876 time to create 1 rle with old method : 0.001399993896484375 length of segment : 40 time for calcul the mask position with numpy : 4.38690185546875e-05 nb_pixel_total : 935 time to create 1 rle with old method : 0.0012357234954833984 length of segment : 42 time for calcul the mask position with numpy : 4.744529724121094e-05 nb_pixel_total : 923 time to create 1 rle with old method : 0.0012373924255371094 length of segment : 40 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -118.19219 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 6.604194641113281e-05 nb_pixel_total : 2150 time to create 1 rle with old method : 0.0025975704193115234 length of segment : 60 time for calcul the mask position with numpy : 0.00012946128845214844 nb_pixel_total : 4494 time to create 1 rle with old method : 0.006846904754638672 length of segment : 60 time for calcul the mask position with numpy : 3.5762786865234375e-05 nb_pixel_total : 108 time to create 1 rle with old method : 0.00019025802612304688 length of segment : 11 time for calcul the mask position with numpy : 3.552436828613281e-05 nb_pixel_total : 253 time to create 1 rle with old method : 0.00033974647521972656 length of segment : 40 time for calcul the mask position with numpy : 3.361701965332031e-05 nb_pixel_total : 118 time to create 1 rle with old method : 0.0002536773681640625 length of segment : 12 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 977 time to create 1 rle with old method : 0.0011975765228271484 length of segment : 71 time for calcul the mask position with numpy : 0.00010418891906738281 nb_pixel_total : 4264 time to create 1 rle with old method : 0.005375385284423828 length of segment : 61 time for calcul the mask position with numpy : 4.482269287109375e-05 nb_pixel_total : 1009 time to create 1 rle with old method : 0.0012316703796386719 length of segment : 42 Processing 1 images image shape: (400, 400, 3) min: 16.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -91.70781 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 5.078315734863281e-05 nb_pixel_total : 1248 time to create 1 rle with old method : 0.0016140937805175781 length of segment : 35 time for calcul the mask position with numpy : 6.890296936035156e-05 nb_pixel_total : 2999 time to create 1 rle with old method : 0.0033712387084960938 length of segment : 84 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -115.36406 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 5.745887756347656e-05 nb_pixel_total : 504 time to create 1 rle with old method : 0.0011119842529296875 length of segment : 28 time for calcul the mask position with numpy : 0.0003223419189453125 nb_pixel_total : 11403 time to create 1 rle with old method : 0.016115665435791016 length of segment : 240 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 147.45547 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 4.267692565917969e-05 nb_pixel_total : 426 time to create 1 rle with old method : 0.0006997585296630859 length of segment : 17 time for calcul the mask position with numpy : 3.528594970703125e-05 nb_pixel_total : 350 time to create 1 rle with old method : 0.0005397796630859375 length of segment : 21 time for calcul the mask position with numpy : 0.0007126331329345703 nb_pixel_total : 61940 time to create 1 rle with old method : 0.0749964714050293 length of segment : 328 time for calcul the mask position with numpy : 0.0005671977996826172 nb_pixel_total : 50442 time to create 1 rle with old method : 0.06440329551696777 length of segment : 255 Processing 1 images image shape: (400, 400, 3) min: 40.00000 max: 229.00000 molded_images shape: (1, 640, 640, 3) min: -69.36406 max: 106.35391 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0014078617095947266 nb_pixel_total : 151369 time to create 1 rle with new method : 0.002174854278564453 length of segment : 397 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -114.83281 max: 149.36953 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 0.00017189979553222656 nb_pixel_total : 9818 time to create 1 rle with old method : 0.012612581253051758 length of segment : 129 time for calcul the mask position with numpy : 4.172325134277344e-05 nb_pixel_total : 396 time to create 1 rle with old method : 0.0006101131439208984 length of segment : 25 time for calcul the mask position with numpy : 5.2928924560546875e-05 nb_pixel_total : 431 time to create 1 rle with old method : 0.0006535053253173828 length of segment : 20 time for calcul the mask position with numpy : 4.57763671875e-05 nb_pixel_total : 656 time to create 1 rle with old method : 0.000997781753540039 length of segment : 31 time for calcul the mask position with numpy : 5.245208740234375e-05 nb_pixel_total : 1666 time to create 1 rle with old method : 0.0022177696228027344 length of segment : 48 time for calcul the mask position with numpy : 3.504753112792969e-05 nb_pixel_total : 158 time to create 1 rle with old method : 0.0002582073211669922 length of segment : 13 time for calcul the mask position with numpy : 4.029273986816406e-05 nb_pixel_total : 303 time to create 1 rle with old method : 0.0005211830139160156 length of segment : 22 time for calcul the mask position with numpy : 3.600120544433594e-05 nb_pixel_total : 321 time to create 1 rle with old method : 0.0005230903625488281 length of segment : 26 Processing 1 images image shape: (400, 400, 3) min: 6.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 7.152557373046875e-05 nb_pixel_total : 416 time to create 1 rle with old method : 0.0008530616760253906 length of segment : 29 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 178 time to create 1 rle with old method : 0.0003085136413574219 length of segment : 20 time for calcul the mask position with numpy : 3.9577484130859375e-05 nb_pixel_total : 139 time to create 1 rle with old method : 0.0002338886260986328 length of segment : 18 time for calcul the mask position with numpy : 5.4836273193359375e-05 nb_pixel_total : 562 time to create 1 rle with old method : 0.0008575916290283203 length of segment : 47 time for calcul the mask position with numpy : 3.886222839355469e-05 nb_pixel_total : 619 time to create 1 rle with old method : 0.0008404254913330078 length of segment : 28 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -115.41875 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 0.000133514404296875 nb_pixel_total : 2270 time to create 1 rle with old method : 0.0040531158447265625 length of segment : 154 time for calcul the mask position with numpy : 0.00012373924255371094 nb_pixel_total : 2212 time to create 1 rle with old method : 0.0027990341186523438 length of segment : 139 time for calcul the mask position with numpy : 6.461143493652344e-05 nb_pixel_total : 852 time to create 1 rle with old method : 0.0011496543884277344 length of segment : 38 time for calcul the mask position with numpy : 5.841255187988281e-05 nb_pixel_total : 609 time to create 1 rle with old method : 0.0010221004486083984 length of segment : 20 time for calcul the mask position with numpy : 8.296966552734375e-05 nb_pixel_total : 1329 time to create 1 rle with old method : 0.0020105838775634766 length of segment : 45 time for calcul the mask position with numpy : 4.839897155761719e-05 nb_pixel_total : 77 time to create 1 rle with old method : 0.0001595020294189453 length of segment : 11 time for calcul the mask position with numpy : 6.318092346191406e-05 nb_pixel_total : 1221 time to create 1 rle with old method : 0.0017802715301513672 length of segment : 44 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 247.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 128.15859 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 10 time for calcul the mask position with numpy : 4.673004150390625e-05 nb_pixel_total : 459 time to create 1 rle with old method : 0.0006301403045654297 length of segment : 39 time for calcul the mask position with numpy : 9.512901306152344e-05 nb_pixel_total : 1397 time to create 1 rle with old method : 0.0019845962524414062 length of segment : 137 time for calcul the mask position with numpy : 0.0002589225769042969 nb_pixel_total : 4824 time to create 1 rle with old method : 0.006113290786743164 length of segment : 220 time for calcul the mask position with numpy : 0.00011515617370605469 nb_pixel_total : 1039 time to create 1 rle with old method : 0.0015366077423095703 length of segment : 96 time for calcul the mask position with numpy : 4.8160552978515625e-05 nb_pixel_total : 296 time to create 1 rle with old method : 0.0004134178161621094 length of segment : 29 time for calcul the mask position with numpy : 6.890296936035156e-05 nb_pixel_total : 1680 time to create 1 rle with old method : 0.0021545886993408203 length of segment : 77 time for calcul the mask position with numpy : 5.91278076171875e-05 nb_pixel_total : 817 time to create 1 rle with old method : 0.0011813640594482422 length of segment : 47 time for calcul the mask position with numpy : 5.245208740234375e-05 nb_pixel_total : 298 time to create 1 rle with old method : 0.0005376338958740234 length of segment : 30 time for calcul the mask position with numpy : 0.00025725364685058594 nb_pixel_total : 4231 time to create 1 rle with old method : 0.005096912384033203 length of segment : 199 time for calcul the mask position with numpy : 6.008148193359375e-05 nb_pixel_total : 706 time to create 1 rle with old method : 0.0009379386901855469 length of segment : 42 Processing 1 images image shape: (280, 400, 3) min: 46.00000 max: 210.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 95.96328 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0011298656463623047 nb_pixel_total : 105065 time to create 1 rle with old method : 0.14147257804870605 length of segment : 282 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 147.84219 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 4.220008850097656e-05 nb_pixel_total : 396 time to create 1 rle with old method : 0.0005514621734619141 length of segment : 43 time for calcul the mask position with numpy : 2.7894973754882812e-05 nb_pixel_total : 70 time to create 1 rle with old method : 0.00010609626770019531 length of segment : 11 time for calcul the mask position with numpy : 3.2901763916015625e-05 nb_pixel_total : 201 time to create 1 rle with old method : 0.0003418922424316406 length of segment : 16 time for calcul the mask position with numpy : 3.409385681152344e-05 nb_pixel_total : 486 time to create 1 rle with old method : 0.000579833984375 length of segment : 32 time for calcul the mask position with numpy : 3.910064697265625e-05 nb_pixel_total : 777 time to create 1 rle with old method : 0.0011410713195800781 length of segment : 22 time for calcul the mask position with numpy : 3.647804260253906e-05 nb_pixel_total : 768 time to create 1 rle with old method : 0.0010597705841064453 length of segment : 19 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 145.52969 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 11 time for calcul the mask position with numpy : 6.031990051269531e-05 nb_pixel_total : 2139 time to create 1 rle with old method : 0.002580404281616211 length of segment : 48 time for calcul the mask position with numpy : 5.793571472167969e-05 nb_pixel_total : 2644 time to create 1 rle with old method : 0.0029973983764648438 length of segment : 60 time for calcul the mask position with numpy : 3.0517578125e-05 nb_pixel_total : 86 time to create 1 rle with old method : 0.0001590251922607422 length of segment : 11 time for calcul the mask position with numpy : 5.0067901611328125e-05 nb_pixel_total : 1309 time to create 1 rle with old method : 0.001550436019897461 length of segment : 51 time for calcul the mask position with numpy : 3.0040740966796875e-05 nb_pixel_total : 290 time to create 1 rle with old method : 0.0003871917724609375 length of segment : 18 time for calcul the mask position with numpy : 4.1484832763671875e-05 nb_pixel_total : 337 time to create 1 rle with old method : 0.0005280971527099609 length of segment : 39 time for calcul the mask position with numpy : 4.5299530029296875e-05 nb_pixel_total : 1011 time to create 1 rle with old method : 0.0012104511260986328 length of segment : 46 time for calcul the mask position with numpy : 4.696846008300781e-05 nb_pixel_total : 1362 time to create 1 rle with old method : 0.0016295909881591797 length of segment : 51 time for calcul the mask position with numpy : 5.125999450683594e-05 nb_pixel_total : 2493 time to create 1 rle with old method : 0.002797842025756836 length of segment : 56 time for calcul the mask position with numpy : 4.1961669921875e-05 nb_pixel_total : 1000 time to create 1 rle with old method : 0.0011515617370605469 length of segment : 47 time for calcul the mask position with numpy : 4.57763671875e-05 nb_pixel_total : 1362 time to create 1 rle with old method : 0.001644134521484375 length of segment : 49 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 149.61953 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 3.266334533691406e-05 nb_pixel_total : 85 time to create 1 rle with old method : 0.00015783309936523438 length of segment : 11 time for calcul the mask position with numpy : 3.337860107421875e-05 nb_pixel_total : 140 time to create 1 rle with old method : 0.0002803802490234375 length of segment : 13 time for calcul the mask position with numpy : 4.696846008300781e-05 nb_pixel_total : 605 time to create 1 rle with old method : 0.0007758140563964844 length of segment : 85 time for calcul the mask position with numpy : 3.24249267578125e-05 nb_pixel_total : 146 time to create 1 rle with old method : 0.00020503997802734375 length of segment : 20 time for calcul the mask position with numpy : 2.8848648071289062e-05 nb_pixel_total : 30 time to create 1 rle with old method : 7.700920104980469e-05 length of segment : 5 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 140.97500 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 5.4836273193359375e-05 nb_pixel_total : 72 time to create 1 rle with old method : 0.0001876354217529297 length of segment : 31 time for calcul the mask position with numpy : 0.0004482269287109375 nb_pixel_total : 45781 time to create 1 rle with old method : 0.04850363731384277 length of segment : 260 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -115.41484 max: 150.14297 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 4.38690185546875e-05 nb_pixel_total : 779 time to create 1 rle with old method : 0.0009632110595703125 length of segment : 29 time for calcul the mask position with numpy : 4.267692565917969e-05 nb_pixel_total : 895 time to create 1 rle with old method : 0.0011436939239501953 length of segment : 30 time for calcul the mask position with numpy : 3.24249267578125e-05 nb_pixel_total : 409 time to create 1 rle with old method : 0.0005254745483398438 length of segment : 30 time for calcul the mask position with numpy : 5.078315734863281e-05 nb_pixel_total : 1898 time to create 1 rle with old method : 0.0022559165954589844 length of segment : 40 time for calcul the mask position with numpy : 4.1484832763671875e-05 nb_pixel_total : 762 time to create 1 rle with old method : 0.0010623931884765625 length of segment : 62 time for calcul the mask position with numpy : 3.1948089599609375e-05 nb_pixel_total : 374 time to create 1 rle with old method : 0.00042724609375 length of segment : 28 time for calcul the mask position with numpy : 4.029273986816406e-05 nb_pixel_total : 772 time to create 1 rle with old method : 0.0010492801666259766 length of segment : 46 time for calcul the mask position with numpy : 4.00543212890625e-05 nb_pixel_total : 1196 time to create 1 rle with old method : 0.001569986343383789 length of segment : 32 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -118.35625 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 6.365776062011719e-05 nb_pixel_total : 2233 time to create 1 rle with old method : 0.0028200149536132812 length of segment : 46 time for calcul the mask position with numpy : 9.012222290039062e-05 nb_pixel_total : 3775 time to create 1 rle with old method : 0.0047872066497802734 length of segment : 58 time for calcul the mask position with numpy : 3.8623809814453125e-05 nb_pixel_total : 347 time to create 1 rle with old method : 0.00045871734619140625 length of segment : 44 time for calcul the mask position with numpy : 3.147125244140625e-05 nb_pixel_total : 111 time to create 1 rle with old method : 0.00018072128295898438 length of segment : 14 Processing 1 images image shape: (400, 400, 3) min: 22.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -89.11797 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 5.984306335449219e-05 nb_pixel_total : 985 time to create 1 rle with old method : 0.0015239715576171875 length of segment : 41 time for calcul the mask position with numpy : 9.965896606445312e-05 nb_pixel_total : 5422 time to create 1 rle with old method : 0.006325244903564453 length of segment : 106 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -108.97344 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 4.839897155761719e-05 nb_pixel_total : 495 time to create 1 rle with old method : 0.0009524822235107422 length of segment : 27 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 148.66250 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 4.100799560546875e-05 nb_pixel_total : 440 time to create 1 rle with old method : 0.0006954669952392578 length of segment : 17 time for calcul the mask position with numpy : 3.3855438232421875e-05 nb_pixel_total : 313 time to create 1 rle with old method : 0.0004150867462158203 length of segment : 21 time for calcul the mask position with numpy : 0.0005815029144287109 nb_pixel_total : 50251 time to create 1 rle with old method : 0.05463886260986328 length of segment : 344 Processing 1 images image shape: (400, 400, 3) min: 40.00000 max: 233.00000 molded_images shape: (1, 640, 640, 3) min: -67.28203 max: 110.62344 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0013015270233154297 nb_pixel_total : 153818 time to create 1 rle with new method : 0.0019659996032714844 length of segment : 403 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.50469 max: 149.62734 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 11 time for calcul the mask position with numpy : 4.982948303222656e-05 nb_pixel_total : 452 time to create 1 rle with old method : 0.0006561279296875 length of segment : 20 time for calcul the mask position with numpy : 3.933906555175781e-05 nb_pixel_total : 214 time to create 1 rle with old method : 0.00034308433532714844 length of segment : 21 time for calcul the mask position with numpy : 4.1961669921875e-05 nb_pixel_total : 216 time to create 1 rle with old method : 0.0003643035888671875 length of segment : 17 time for calcul the mask position with numpy : 0.0001823902130126953 nb_pixel_total : 9458 time to create 1 rle with old method : 0.01178431510925293 length of segment : 129 time for calcul the mask position with numpy : 3.504753112792969e-05 nb_pixel_total : 165 time to create 1 rle with old method : 0.0002770423889160156 length of segment : 13 time for calcul the mask position with numpy : 6.246566772460938e-05 nb_pixel_total : 1661 time to create 1 rle with old method : 0.00222015380859375 length of segment : 48 time for calcul the mask position with numpy : 4.9591064453125e-05 nb_pixel_total : 459 time to create 1 rle with old method : 0.0007405281066894531 length of segment : 33 time for calcul the mask position with numpy : 5.6743621826171875e-05 nb_pixel_total : 1499 time to create 1 rle with old method : 0.0021805763244628906 length of segment : 35 time for calcul the mask position with numpy : 4.863739013671875e-05 nb_pixel_total : 1217 time to create 1 rle with old method : 0.001558065414428711 length of segment : 43 time for calcul the mask position with numpy : 3.170967102050781e-05 nb_pixel_total : 105 time to create 1 rle with old method : 0.00015854835510253906 length of segment : 11 time for calcul the mask position with numpy : 5.0067901611328125e-05 nb_pixel_total : 1311 time to create 1 rle with old method : 0.0017848014831542969 length of segment : 32 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -121.12969 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 4.6253204345703125e-05 nb_pixel_total : 407 time to create 1 rle with old method : 0.0005543231964111328 length of segment : 28 time for calcul the mask position with numpy : 0.00026035308837890625 nb_pixel_total : 12524 time to create 1 rle with old method : 0.014102935791015625 length of segment : 106 time for calcul the mask position with numpy : 3.6716461181640625e-05 nb_pixel_total : 244 time to create 1 rle with old method : 0.0003390312194824219 length of segment : 25 time for calcul the mask position with numpy : 2.9802322387695312e-05 nb_pixel_total : 91 time to create 1 rle with old method : 0.000148773193359375 length of segment : 11 time for calcul the mask position with numpy : 4.744529724121094e-05 nb_pixel_total : 816 time to create 1 rle with old method : 0.0011258125305175781 length of segment : 43 time for calcul the mask position with numpy : 3.266334533691406e-05 nb_pixel_total : 218 time to create 1 rle with old method : 0.00032520294189453125 length of segment : 19 time for calcul the mask position with numpy : 3.790855407714844e-05 nb_pixel_total : 659 time to create 1 rle with old method : 0.0008084774017333984 length of segment : 28 time for calcul the mask position with numpy : 3.457069396972656e-05 nb_pixel_total : 368 time to create 1 rle with old method : 0.00047779083251953125 length of segment : 22 time for calcul the mask position with numpy : 0.00018715858459472656 nb_pixel_total : 12133 time to create 1 rle with old method : 0.013661861419677734 length of segment : 108 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -119.18047 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 10 time for calcul the mask position with numpy : 5.1975250244140625e-05 nb_pixel_total : 578 time to create 1 rle with old method : 0.0008313655853271484 length of segment : 19 time for calcul the mask position with numpy : 4.744529724121094e-05 nb_pixel_total : 1396 time to create 1 rle with old method : 0.001631021499633789 length of segment : 42 time for calcul the mask position with numpy : 8.678436279296875e-05 nb_pixel_total : 1672 time to create 1 rle with old method : 0.002037525177001953 length of segment : 130 time for calcul the mask position with numpy : 0.00010347366333007812 nb_pixel_total : 1900 time to create 1 rle with old method : 0.002498626708984375 length of segment : 93 time for calcul the mask position with numpy : 5.221366882324219e-05 nb_pixel_total : 1465 time to create 1 rle with old method : 0.001825571060180664 length of segment : 37 time for calcul the mask position with numpy : 5.6743621826171875e-05 nb_pixel_total : 1246 time to create 1 rle with old method : 0.0016696453094482422 length of segment : 45 time for calcul the mask position with numpy : 0.00010037422180175781 nb_pixel_total : 2221 time to create 1 rle with old method : 0.002663135528564453 length of segment : 100 time for calcul the mask position with numpy : 4.100799560546875e-05 nb_pixel_total : 801 time to create 1 rle with old method : 0.0010221004486083984 length of segment : 37 time for calcul the mask position with numpy : 5.2928924560546875e-05 nb_pixel_total : 1131 time to create 1 rle with old method : 0.0015194416046142578 length of segment : 44 time for calcul the mask position with numpy : 3.933906555175781e-05 nb_pixel_total : 869 time to create 1 rle with old method : 0.0010426044464111328 length of segment : 38 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 240.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 126.81094 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 393 time to create 1 rle with old method : 0.0005025863647460938 length of segment : 37 time for calcul the mask position with numpy : 3.62396240234375e-05 nb_pixel_total : 410 time to create 1 rle with old method : 0.0005295276641845703 length of segment : 37 time for calcul the mask position with numpy : 4.363059997558594e-05 nb_pixel_total : 726 time to create 1 rle with old method : 0.0008990764617919922 length of segment : 42 time for calcul the mask position with numpy : 4.172325134277344e-05 nb_pixel_total : 819 time to create 1 rle with old method : 0.0010266304016113281 length of segment : 45 time for calcul the mask position with numpy : 3.0517578125e-05 nb_pixel_total : 87 time to create 1 rle with old method : 0.00020837783813476562 length of segment : 15 time for calcul the mask position with numpy : 0.0001666545867919922 nb_pixel_total : 4821 time to create 1 rle with old method : 0.005837440490722656 length of segment : 225 time for calcul the mask position with numpy : 3.981590270996094e-05 nb_pixel_total : 262 time to create 1 rle with old method : 0.0003509521484375 length of segment : 30 time for calcul the mask position with numpy : 7.081031799316406e-05 nb_pixel_total : 1522 time to create 1 rle with old method : 0.0017995834350585938 length of segment : 81 time for calcul the mask position with numpy : 5.4836273193359375e-05 nb_pixel_total : 1520 time to create 1 rle with old method : 0.0018117427825927734 length of segment : 73 Processing 1 images image shape: (280, 400, 3) min: 40.00000 max: 215.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 97.23672 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0012640953063964844 nb_pixel_total : 106010 time to create 1 rle with old method : 0.15273475646972656 length of segment : 282 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 149.13125 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 5.054473876953125e-05 nb_pixel_total : 217 time to create 1 rle with old method : 0.00048160552978515625 length of segment : 14 time for calcul the mask position with numpy : 7.367134094238281e-05 nb_pixel_total : 500 time to create 1 rle with old method : 0.0007653236389160156 length of segment : 44 time for calcul the mask position with numpy : 3.0279159545898438e-05 nb_pixel_total : 73 time to create 1 rle with old method : 0.00012421607971191406 length of segment : 11 time for calcul the mask position with numpy : 3.2901763916015625e-05 nb_pixel_total : 290 time to create 1 rle with old method : 0.00040912628173828125 length of segment : 22 time for calcul the mask position with numpy : 5.841255187988281e-05 nb_pixel_total : 1690 time to create 1 rle with old method : 0.001972198486328125 length of segment : 92 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 143.14297 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 4.57763671875e-05 nb_pixel_total : 338 time to create 1 rle with old method : 0.0005419254302978516 length of segment : 39 time for calcul the mask position with numpy : 7.033348083496094e-05 nb_pixel_total : 2749 time to create 1 rle with old method : 0.0031898021697998047 length of segment : 86 time for calcul the mask position with numpy : 3.218650817871094e-05 nb_pixel_total : 109 time to create 1 rle with old method : 0.00017571449279785156 length of segment : 14 time for calcul the mask position with numpy : 5.4836273193359375e-05 nb_pixel_total : 2130 time to create 1 rle with old method : 0.0025408267974853516 length of segment : 47 time for calcul the mask position with numpy : 4.291534423828125e-05 nb_pixel_total : 996 time to create 1 rle with old method : 0.001172780990600586 length of segment : 45 time for calcul the mask position with numpy : 5.269050598144531e-05 nb_pixel_total : 2014 time to create 1 rle with old method : 0.002401590347290039 length of segment : 46 time for calcul the mask position with numpy : 4.8160552978515625e-05 nb_pixel_total : 966 time to create 1 rle with old method : 0.0012180805206298828 length of segment : 69 time for calcul the mask position with numpy : 3.0517578125e-05 nb_pixel_total : 111 time to create 1 rle with old method : 0.00016951560974121094 length of segment : 13 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.33828 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 5.9604644775390625e-05 nb_pixel_total : 272 time to create 1 rle with old method : 0.00039005279541015625 length of segment : 48 time for calcul the mask position with numpy : 3.1948089599609375e-05 nb_pixel_total : 158 time to create 1 rle with old method : 0.0002193450927734375 length of segment : 20 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 117 time to create 1 rle with old method : 0.00024366378784179688 length of segment : 36 time for calcul the mask position with numpy : 3.266334533691406e-05 nb_pixel_total : 226 time to create 1 rle with old method : 0.0003190040588378906 length of segment : 15 time for calcul the mask position with numpy : 3.0040740966796875e-05 nb_pixel_total : 96 time to create 1 rle with old method : 0.00018310546875 length of segment : 12 time for calcul the mask position with numpy : 2.8848648071289062e-05 nb_pixel_total : 39 time to create 1 rle with old method : 8.96453857421875e-05 length of segment : 6 time for calcul the mask position with numpy : 2.8848648071289062e-05 nb_pixel_total : 75 time to create 1 rle with old method : 0.00013780593872070312 length of segment : 8 time for calcul the mask position with numpy : 3.170967102050781e-05 nb_pixel_total : 196 time to create 1 rle with old method : 0.0002770423889160156 length of segment : 23 time for calcul the mask position with numpy : 2.8848648071289062e-05 nb_pixel_total : 29 time to create 1 rle with old method : 7.2479248046875e-05 length of segment : 5 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 142.35000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 4.100799560546875e-05 nb_pixel_total : 607 time to create 1 rle with old method : 0.0007741451263427734 length of segment : 34 time for calcul the mask position with numpy : 0.00048804283142089844 nb_pixel_total : 44733 time to create 1 rle with old method : 0.04862356185913086 length of segment : 286 time for calcul the mask position with numpy : 6.079673767089844e-05 nb_pixel_total : 4 time to create 1 rle with old method : 2.3365020751953125e-05 length of segment : 4 Detection mask done ! Trying to reset tf kernel 887942 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 4686 tf kernel not reseted sub process len(results) : 237 len(list_Values) 237 None max_time_sub_proc : 3600 parent process len(results) : 0 len(list_Values) 237 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 9861 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Catched exception ! Connect or reconnect ! Number saved : None batch 1 Loaded 237 chid ids of type : 4228 Number RLEs to save : 14344 save missing photos in datou_result : time spend for datou_step_exec : 25.059391498565674 time spend to save output : 3.788194179534912 total time spend for step 1 : 28.847585678100586 step2:brightness Mon Feb 3 01:20:57 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1738542027_887412_1333956514_bc747c10d5a1d9d3fe6063a2693ae234.jpg treat image : temp/1738542027_887412_1333956513_dff299f7f3554d874166d7c102932605.jpg treat image : temp/1738542027_887412_1333956512_d0c13d704c6bc996807ebae7eea50ad3.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 3 time used for this insertion : 0.012023687362670898 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 3 time used for this insertion : 0.21952176094055176 save missing photos in datou_result : time spend for datou_step_exec : 1.2513129711151123 time spend to save output : 0.23630332946777344 total time spend for step 2 : 1.4876163005828857 step3:blur_detection Mon Feb 3 01:20:58 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1738542027_887412_1333956514_bc747c10d5a1d9d3fe6063a2693ae234.jpg resize: (1080, 1920) 1333956514 -7.645552774554034 treat image : temp/1738542027_887412_1333956513_dff299f7f3554d874166d7c102932605.jpg resize: (1080, 1920) 1333956513 -7.689588777369138 treat image : temp/1738542027_887412_1333956512_d0c13d704c6bc996807ebae7eea50ad3.jpg resize: (1080, 1920) 1333956512 -7.709285482234909 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 3 time used for this insertion : 0.011673688888549805 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 3 time used for this insertion : 0.012951135635375977 save missing photos in datou_result : time spend for datou_step_exec : 3.1331262588500977 time spend to save output : 0.028905391693115234 total time spend for step 3 : 3.162031650543213 step4:rle_unique_nms_with_priority Mon Feb 3 01:21:01 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 237 chid ids of type : 4228 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 78 nb_hashtags : 6 time to prepare the origin masks : 1.0847654342651367 time for calcul the mask position with numpy : 0.04833245277404785 nb_pixel_total : 1725559 time to create 1 rle with new method : 0.5447561740875244 time for calcul the mask position with numpy : 0.008559226989746094 nb_pixel_total : 2136 time to create 1 rle with old method : 0.002463102340698242 time for calcul the mask position with numpy : 0.008458852767944336 nb_pixel_total : 370 time to create 1 rle with old method : 0.00040602684020996094 time for calcul the mask position with numpy : 0.008487462997436523 nb_pixel_total : 121 time to create 1 rle with old method : 0.00016832351684570312 time for calcul the mask position with numpy : 0.008470296859741211 nb_pixel_total : 880 time to create 1 rle with old method : 0.0009851455688476562 time for calcul the mask position with numpy : 0.008474111557006836 nb_pixel_total : 60 time to create 1 rle with old method : 0.00010967254638671875 time for calcul the mask position with numpy : 0.008479118347167969 nb_pixel_total : 226 time to create 1 rle with old method : 0.0002646446228027344 time for calcul the mask position with numpy : 0.008481502532958984 nb_pixel_total : 816 time to create 1 rle with old method : 0.0009396076202392578 time for calcul the mask position with numpy : 0.008448600769042969 nb_pixel_total : 8 time to create 1 rle with old method : 2.9325485229492188e-05 time for calcul the mask position with numpy : 0.009175539016723633 nb_pixel_total : 2372 time to create 1 rle with old method : 0.002495288848876953 time for calcul the mask position with numpy : 0.007836341857910156 nb_pixel_total : 1011 time to create 1 rle with old method : 0.0011134147644042969 time for calcul the mask position with numpy : 0.007808685302734375 nb_pixel_total : 126 time to create 1 rle with old method : 0.00014543533325195312 time for calcul the mask position with numpy : 0.007899999618530273 nb_pixel_total : 264 time to create 1 rle with old method : 0.00029087066650390625 time for calcul the mask position with numpy : 0.008333206176757812 nb_pixel_total : 3781 time to create 1 rle with old method : 0.004316568374633789 time for calcul the mask position with numpy : 0.008474588394165039 nb_pixel_total : 871 time to create 1 rle with old method : 0.0009491443634033203 time for calcul the mask position with numpy : 0.00845026969909668 nb_pixel_total : 757 time to create 1 rle with old method : 0.0008251667022705078 time for calcul the mask position with numpy : 0.008447647094726562 nb_pixel_total : 530 time to create 1 rle with old method : 0.0006051063537597656 time for calcul the mask position with numpy : 0.00844430923461914 nb_pixel_total : 4842 time to create 1 rle with old method : 0.005175352096557617 time for calcul the mask position with numpy : 0.008159637451171875 nb_pixel_total : 1507 time to create 1 rle with old method : 0.0016562938690185547 time for calcul the mask position with numpy : 0.00988006591796875 nb_pixel_total : 6333 time to create 1 rle with old method : 0.0073359012603759766 time for calcul the mask position with numpy : 0.008313417434692383 nb_pixel_total : 323 time to create 1 rle with old method : 0.0003299713134765625 time for calcul the mask position with numpy : 0.008348703384399414 nb_pixel_total : 400 time to create 1 rle with old method : 0.000415802001953125 time for calcul the mask position with numpy : 0.009412050247192383 nb_pixel_total : 152768 time to create 1 rle with new method : 0.0722053050994873 time for calcul the mask position with numpy : 0.008484601974487305 nb_pixel_total : 425 time to create 1 rle with old method : 0.0005450248718261719 time for calcul the mask position with numpy : 0.008431434631347656 nb_pixel_total : 1690 time to create 1 rle with old method : 0.0018796920776367188 time for calcul the mask position with numpy : 0.008478164672851562 nb_pixel_total : 667 time to create 1 rle with old method : 0.0007631778717041016 time for calcul the mask position with numpy : 0.008247137069702148 nb_pixel_total : 807 time to create 1 rle with old method : 0.0008432865142822266 time for calcul the mask position with numpy : 0.008123397827148438 nb_pixel_total : 73 time to create 1 rle with old method : 0.00011467933654785156 time for calcul the mask position with numpy : 0.008397102355957031 nb_pixel_total : 1262 time to create 1 rle with old method : 0.0013532638549804688 time for calcul the mask position with numpy : 0.00845479965209961 nb_pixel_total : 821 time to create 1 rle with old method : 0.0009047985076904297 time for calcul the mask position with numpy : 0.008538484573364258 nb_pixel_total : 12644 time to create 1 rle with old method : 0.013615608215332031 time for calcul the mask position with numpy : 0.008520126342773438 nb_pixel_total : 293 time to create 1 rle with old method : 0.00034809112548828125 time for calcul the mask position with numpy : 0.008312702178955078 nb_pixel_total : 193 time to create 1 rle with old method : 0.0002994537353515625 time for calcul the mask position with numpy : 0.00812840461730957 nb_pixel_total : 585 time to create 1 rle with old method : 0.0006794929504394531 time for calcul the mask position with numpy : 0.008142709732055664 nb_pixel_total : 10434 time to create 1 rle with old method : 0.01104426383972168 time for calcul the mask position with numpy : 0.008167505264282227 nb_pixel_total : 167 time to create 1 rle with old method : 0.00029921531677246094 time for calcul the mask position with numpy : 0.00814962387084961 nb_pixel_total : 721 time to create 1 rle with old method : 0.0007877349853515625 time for calcul the mask position with numpy : 0.008260726928710938 nb_pixel_total : 19 time to create 1 rle with old method : 4.673004150390625e-05 time for calcul the mask position with numpy : 0.008177042007446289 nb_pixel_total : 4973 time to create 1 rle with old method : 0.005491733551025391 time for calcul the mask position with numpy : 0.008454322814941406 nb_pixel_total : 676 time to create 1 rle with old method : 0.0007865428924560547 time for calcul the mask position with numpy : 0.008447408676147461 nb_pixel_total : 2603 time to create 1 rle with old method : 0.0028138160705566406 time for calcul the mask position with numpy : 0.008435249328613281 nb_pixel_total : 178 time to create 1 rle with old method : 0.00026035308837890625 time for calcul the mask position with numpy : 0.00841522216796875 nb_pixel_total : 1514 time to create 1 rle with old method : 0.0017158985137939453 time for calcul the mask position with numpy : 0.008439779281616211 nb_pixel_total : 795 time to create 1 rle with old method : 0.0008866786956787109 time for calcul the mask position with numpy : 0.008499622344970703 nb_pixel_total : 203 time to create 1 rle with old method : 0.0002357959747314453 time for calcul the mask position with numpy : 0.008528709411621094 nb_pixel_total : 233 time to create 1 rle with old method : 0.0002605915069580078 time for calcul the mask position with numpy : 0.008365392684936523 nb_pixel_total : 1446 time to create 1 rle with old method : 0.0016069412231445312 time for calcul the mask position with numpy : 0.008396387100219727 nb_pixel_total : 1228 time to create 1 rle with old method : 0.0013654232025146484 time for calcul the mask position with numpy : 0.008463382720947266 nb_pixel_total : 508 time to create 1 rle with old method : 0.0005600452423095703 time for calcul the mask position with numpy : 0.008468389511108398 nb_pixel_total : 37 time to create 1 rle with old method : 6.127357482910156e-05 time for calcul the mask position with numpy : 0.00824117660522461 nb_pixel_total : 157 time to create 1 rle with old method : 0.00017142295837402344 time for calcul the mask position with numpy : 0.007766008377075195 nb_pixel_total : 35 time to create 1 rle with old method : 6.866455078125e-05 time for calcul the mask position with numpy : 0.008535146713256836 nb_pixel_total : 1620 time to create 1 rle with old method : 0.0018308162689208984 time for calcul the mask position with numpy : 0.008951902389526367 nb_pixel_total : 157 time to create 1 rle with old method : 0.00017976760864257812 time for calcul the mask position with numpy : 0.008340120315551758 nb_pixel_total : 722 time to create 1 rle with old method : 0.000823974609375 time for calcul the mask position with numpy : 0.008440494537353516 nb_pixel_total : 421 time to create 1 rle with old method : 0.00047659873962402344 time for calcul the mask position with numpy : 0.008449792861938477 nb_pixel_total : 383 time to create 1 rle with old method : 0.00042748451232910156 time for calcul the mask position with numpy : 0.008435249328613281 nb_pixel_total : 1656 time to create 1 rle with old method : 0.0018029212951660156 time for calcul the mask position with numpy : 0.008462905883789062 nb_pixel_total : 448 time to create 1 rle with old method : 0.0004942417144775391 time for calcul the mask position with numpy : 0.008451700210571289 nb_pixel_total : 217 time to create 1 rle with old method : 0.0002849102020263672 time for calcul the mask position with numpy : 0.00896143913269043 nb_pixel_total : 104543 time to create 1 rle with old method : 0.11576557159423828 time for calcul the mask position with numpy : 0.008489608764648438 nb_pixel_total : 893 time to create 1 rle with old method : 0.0010251998901367188 time for calcul the mask position with numpy : 0.008417844772338867 nb_pixel_total : 368 time to create 1 rle with old method : 0.00044274330139160156 time for calcul the mask position with numpy : 0.008388519287109375 nb_pixel_total : 1191 time to create 1 rle with old method : 0.0013353824615478516 time for calcul the mask position with numpy : 0.008423328399658203 nb_pixel_total : 1407 time to create 1 rle with old method : 0.0015254020690917969 time for calcul the mask position with numpy : 0.00844883918762207 nb_pixel_total : 472 time to create 1 rle with old method : 0.0005252361297607422 time for calcul the mask position with numpy : 0.00843358039855957 nb_pixel_total : 318 time to create 1 rle with old method : 0.00037360191345214844 time for calcul the mask position with numpy : 0.008286476135253906 nb_pixel_total : 3360 time to create 1 rle with old method : 0.003661632537841797 time for calcul the mask position with numpy : 0.008470296859741211 nb_pixel_total : 74 time to create 1 rle with old method : 9.608268737792969e-05 time for calcul the mask position with numpy : 0.00847768783569336 nb_pixel_total : 142 time to create 1 rle with old method : 0.00016736984252929688 time for calcul the mask position with numpy : 0.008629083633422852 nb_pixel_total : 164 time to create 1 rle with old method : 0.00019550323486328125 time for calcul the mask position with numpy : 0.008749723434448242 nb_pixel_total : 65 time to create 1 rle with old method : 0.00011277198791503906 time for calcul the mask position with numpy : 0.008614540100097656 nb_pixel_total : 8 time to create 1 rle with old method : 3.719329833984375e-05 time for calcul the mask position with numpy : 0.008508920669555664 nb_pixel_total : 28 time to create 1 rle with old method : 5.054473876953125e-05 time for calcul the mask position with numpy : 0.008445262908935547 nb_pixel_total : 100 time to create 1 rle with old method : 0.00013327598571777344 time for calcul the mask position with numpy : 0.00913858413696289 nb_pixel_total : 901 time to create 1 rle with old method : 0.0014791488647460938 time for calcul the mask position with numpy : 0.010352373123168945 nb_pixel_total : 2151 time to create 1 rle with old method : 0.003263711929321289 time for calcul the mask position with numpy : 0.010523796081542969 nb_pixel_total : 70 time to create 1 rle with old method : 0.00010251998901367188 time for calcul the mask position with numpy : 0.008911609649658203 nb_pixel_total : 273 time to create 1 rle with old method : 0.00032448768615722656 create new chi : 1.5643534660339355 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.01615619659423828 batch 1 Loaded 79 chid ids of type : 4230 Number RLEs to save : 9067 TO DO : save crop sub photo not yet done ! save time : 1.1898243427276611 nb_obj : 75 nb_hashtags : 6 time to prepare the origin masks : 1.8976154327392578 time for calcul the mask position with numpy : 0.03090524673461914 nb_pixel_total : 1635714 time to create 1 rle with new method : 0.08265042304992676 time for calcul the mask position with numpy : 0.010292768478393555 nb_pixel_total : 61940 time to create 1 rle with old method : 0.06929492950439453 time for calcul the mask position with numpy : 0.0066988468170166016 nb_pixel_total : 164 time to create 1 rle with old method : 0.000232696533203125 time for calcul the mask position with numpy : 0.006342411041259766 nb_pixel_total : 876 time to create 1 rle with old method : 0.0009846687316894531 time for calcul the mask position with numpy : 0.006166696548461914 nb_pixel_total : 154 time to create 1 rle with old method : 0.0003063678741455078 time for calcul the mask position with numpy : 0.0069255828857421875 nb_pixel_total : 977 time to create 1 rle with old method : 0.0011165142059326172 time for calcul the mask position with numpy : 0.00687861442565918 nb_pixel_total : 923 time to create 1 rle with old method : 0.0010609626770019531 time for calcul the mask position with numpy : 0.0060503482818603516 nb_pixel_total : 253 time to create 1 rle with old method : 0.0002968311309814453 time for calcul the mask position with numpy : 0.0060231685638427734 nb_pixel_total : 2150 time to create 1 rle with old method : 0.0024809837341308594 time for calcul the mask position with numpy : 0.005841255187988281 nb_pixel_total : 11403 time to create 1 rle with old method : 0.012292623519897461 time for calcul the mask position with numpy : 0.0058269500732421875 nb_pixel_total : 108 time to create 1 rle with old method : 0.00013494491577148438 time for calcul the mask position with numpy : 0.00584864616394043 nb_pixel_total : 15 time to create 1 rle with old method : 5.412101745605469e-05 time for calcul the mask position with numpy : 0.005688667297363281 nb_pixel_total : 288 time to create 1 rle with old method : 0.0004162788391113281 time for calcul the mask position with numpy : 0.005962848663330078 nb_pixel_total : 4494 time to create 1 rle with old method : 0.0049703121185302734 time for calcul the mask position with numpy : 0.0059888362884521484 nb_pixel_total : 2010 time to create 1 rle with old method : 0.0022630691528320312 time for calcul the mask position with numpy : 0.005985736846923828 nb_pixel_total : 504 time to create 1 rle with old method : 0.0006155967712402344 time for calcul the mask position with numpy : 0.006134986877441406 nb_pixel_total : 2999 time to create 1 rle with old method : 0.0032536983489990234 time for calcul the mask position with numpy : 0.00639033317565918 nb_pixel_total : 350 time to create 1 rle with old method : 0.0003972053527832031 time for calcul the mask position with numpy : 0.00650787353515625 nb_pixel_total : 1248 time to create 1 rle with old method : 0.001458883285522461 time for calcul the mask position with numpy : 0.007502079010009766 nb_pixel_total : 426 time to create 1 rle with old method : 0.0005130767822265625 time for calcul the mask position with numpy : 0.008268117904663086 nb_pixel_total : 151369 time to create 1 rle with new method : 0.03808283805847168 time for calcul the mask position with numpy : 0.0059473514556884766 nb_pixel_total : 2212 time to create 1 rle with old method : 0.002496957778930664 time for calcul the mask position with numpy : 0.005580902099609375 nb_pixel_total : 158 time to create 1 rle with old method : 0.000179290771484375 time for calcul the mask position with numpy : 0.005902767181396484 nb_pixel_total : 619 time to create 1 rle with old method : 0.0006740093231201172 time for calcul the mask position with numpy : 0.00731205940246582 nb_pixel_total : 303 time to create 1 rle with old method : 0.00036454200744628906 time for calcul the mask position with numpy : 0.009773731231689453 nb_pixel_total : 27 time to create 1 rle with old method : 8.606910705566406e-05 time for calcul the mask position with numpy : 0.010483980178833008 nb_pixel_total : 852 time to create 1 rle with old method : 0.0009267330169677734 time for calcul the mask position with numpy : 0.009997367858886719 nb_pixel_total : 817 time to create 1 rle with old method : 0.0009105205535888672 time for calcul the mask position with numpy : 0.009920120239257812 nb_pixel_total : 8 time to create 1 rle with old method : 3.123283386230469e-05 time for calcul the mask position with numpy : 0.010158777236938477 nb_pixel_total : 609 time to create 1 rle with old method : 0.0006906986236572266 time for calcul the mask position with numpy : 0.010523319244384766 nb_pixel_total : 296 time to create 1 rle with old method : 0.0003647804260253906 time for calcul the mask position with numpy : 0.010547399520874023 nb_pixel_total : 9818 time to create 1 rle with old method : 0.010613679885864258 time for calcul the mask position with numpy : 0.010842561721801758 nb_pixel_total : 77 time to create 1 rle with old method : 0.00011372566223144531 time for calcul the mask position with numpy : 0.009893417358398438 nb_pixel_total : 706 time to create 1 rle with old method : 0.0007412433624267578 time for calcul the mask position with numpy : 0.009960174560546875 nb_pixel_total : 148 time to create 1 rle with old method : 0.00023293495178222656 time for calcul the mask position with numpy : 0.010237455368041992 nb_pixel_total : 1397 time to create 1 rle with old method : 0.0016293525695800781 time for calcul the mask position with numpy : 0.010332822799682617 nb_pixel_total : 4824 time to create 1 rle with old method : 0.0051975250244140625 time for calcul the mask position with numpy : 0.009792089462280273 nb_pixel_total : 2270 time to create 1 rle with old method : 0.0026879310607910156 time for calcul the mask position with numpy : 0.009857177734375 nb_pixel_total : 176 time to create 1 rle with old method : 0.0002665519714355469 time for calcul the mask position with numpy : 0.009608983993530273 nb_pixel_total : 562 time to create 1 rle with old method : 0.000629425048828125 time for calcul the mask position with numpy : 0.0060443878173828125 nb_pixel_total : 139 time to create 1 rle with old method : 0.00017380714416503906 time for calcul the mask position with numpy : 0.005999088287353516 nb_pixel_total : 178 time to create 1 rle with old method : 0.0002181529998779297 time for calcul the mask position with numpy : 0.005974769592285156 nb_pixel_total : 1329 time to create 1 rle with old method : 0.001512289047241211 time for calcul the mask position with numpy : 0.006486415863037109 nb_pixel_total : 12 time to create 1 rle with old method : 8.606910705566406e-05 time for calcul the mask position with numpy : 0.005964040756225586 nb_pixel_total : 459 time to create 1 rle with old method : 0.0005135536193847656 time for calcul the mask position with numpy : 0.005972862243652344 nb_pixel_total : 1680 time to create 1 rle with old method : 0.0018777847290039062 time for calcul the mask position with numpy : 0.006001472473144531 nb_pixel_total : 431 time to create 1 rle with old method : 0.0004944801330566406 time for calcul the mask position with numpy : 0.005976200103759766 nb_pixel_total : 656 time to create 1 rle with old method : 0.0007417201995849609 time for calcul the mask position with numpy : 0.0059719085693359375 nb_pixel_total : 1666 time to create 1 rle with old method : 0.0018429756164550781 time for calcul the mask position with numpy : 0.00604248046875 nb_pixel_total : 416 time to create 1 rle with old method : 0.0004725456237792969 time for calcul the mask position with numpy : 0.006005287170410156 nb_pixel_total : 396 time to create 1 rle with old method : 0.000461578369140625 time for calcul the mask position with numpy : 0.006796598434448242 nb_pixel_total : 105065 time to create 1 rle with old method : 0.11048102378845215 time for calcul the mask position with numpy : 0.006019115447998047 nb_pixel_total : 54 time to create 1 rle with old method : 0.00014853477478027344 time for calcul the mask position with numpy : 0.005936145782470703 nb_pixel_total : 777 time to create 1 rle with old method : 0.00090789794921875 time for calcul the mask position with numpy : 0.005969047546386719 nb_pixel_total : 337 time to create 1 rle with old method : 0.00040531158447265625 time for calcul the mask position with numpy : 0.006348371505737305 nb_pixel_total : 72 time to create 1 rle with old method : 0.000118255615234375 time for calcul the mask position with numpy : 0.006227970123291016 nb_pixel_total : 45781 time to create 1 rle with old method : 0.04943227767944336 time for calcul the mask position with numpy : 0.006430387496948242 nb_pixel_total : 396 time to create 1 rle with old method : 0.0004999637603759766 time for calcul the mask position with numpy : 0.006100654602050781 nb_pixel_total : 17 time to create 1 rle with old method : 7.62939453125e-05 time for calcul the mask position with numpy : 0.006357431411743164 nb_pixel_total : 168 time to create 1 rle with old method : 0.0002357959747314453 time for calcul the mask position with numpy : 0.006081819534301758 nb_pixel_total : 1309 time to create 1 rle with old method : 0.001386404037475586 time for calcul the mask position with numpy : 0.005707263946533203 nb_pixel_total : 290 time to create 1 rle with old method : 0.0003459453582763672 time for calcul the mask position with numpy : 0.00924062728881836 nb_pixel_total : 201 time to create 1 rle with old method : 0.0002579689025878906 time for calcul the mask position with numpy : 0.0061037540435791016 nb_pixel_total : 2644 time to create 1 rle with old method : 0.002792835235595703 time for calcul the mask position with numpy : 0.005780220031738281 nb_pixel_total : 31 time to create 1 rle with old method : 9.226799011230469e-05 time for calcul the mask position with numpy : 0.00652313232421875 nb_pixel_total : 85 time to create 1 rle with old method : 0.00012612342834472656 time for calcul the mask position with numpy : 0.0059947967529296875 nb_pixel_total : 146 time to create 1 rle with old method : 0.00018095970153808594 time for calcul the mask position with numpy : 0.005972146987915039 nb_pixel_total : 70 time to create 1 rle with old method : 9.751319885253906e-05 time for calcul the mask position with numpy : 0.0059697628021240234 nb_pixel_total : 605 time to create 1 rle with old method : 0.000705718994140625 time for calcul the mask position with numpy : 0.006021261215209961 nb_pixel_total : 486 time to create 1 rle with old method : 0.0005533695220947266 time for calcul the mask position with numpy : 0.005966663360595703 nb_pixel_total : 30 time to create 1 rle with old method : 0.00011467933654785156 time for calcul the mask position with numpy : 0.005978584289550781 nb_pixel_total : 54 time to create 1 rle with old method : 0.0001361370086669922 time for calcul the mask position with numpy : 0.006006956100463867 nb_pixel_total : 2139 time to create 1 rle with old method : 0.002361774444580078 time for calcul the mask position with numpy : 0.005987405776977539 nb_pixel_total : 1011 time to create 1 rle with old method : 0.0011157989501953125 time for calcul the mask position with numpy : 0.005953788757324219 nb_pixel_total : 140 time to create 1 rle with old method : 0.0001881122589111328 time for calcul the mask position with numpy : 0.006006479263305664 nb_pixel_total : 86 time to create 1 rle with old method : 0.00011920928955078125 create new chi : 0.9965450763702393 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.0011305809020996094 batch 1 Loaded 76 chid ids of type : 4230 Number RLEs to save : 9688 TO DO : save crop sub photo not yet done ! save time : 5.243853330612183 nb_obj : 84 nb_hashtags : 6 time to prepare the origin masks : 1.4321515560150146 time for calcul the mask position with numpy : 0.5202240943908691 nb_pixel_total : 1641322 time to create 1 rle with new method : 0.08110332489013672 time for calcul the mask position with numpy : 0.006242275238037109 nb_pixel_total : 347 time to create 1 rle with old method : 0.0004241466522216797 time for calcul the mask position with numpy : 0.006551504135131836 nb_pixel_total : 762 time to create 1 rle with old method : 0.0013537406921386719 time for calcul the mask position with numpy : 0.006978034973144531 nb_pixel_total : 409 time to create 1 rle with old method : 0.00048828125 time for calcul the mask position with numpy : 0.010068416595458984 nb_pixel_total : 147 time to create 1 rle with old method : 0.000213623046875 time for calcul the mask position with numpy : 0.006936073303222656 nb_pixel_total : 6 time to create 1 rle with old method : 2.2172927856445312e-05 time for calcul the mask position with numpy : 0.006198406219482422 nb_pixel_total : 130 time to create 1 rle with old method : 0.00018024444580078125 time for calcul the mask position with numpy : 0.006051540374755859 nb_pixel_total : 1898 time to create 1 rle with old method : 0.002094268798828125 time for calcul the mask position with numpy : 0.00618433952331543 nb_pixel_total : 779 time to create 1 rle with old method : 0.0008838176727294922 time for calcul the mask position with numpy : 0.006345033645629883 nb_pixel_total : 50251 time to create 1 rle with old method : 0.053617000579833984 time for calcul the mask position with numpy : 0.00665593147277832 nb_pixel_total : 936 time to create 1 rle with old method : 0.000982046127319336 time for calcul the mask position with numpy : 0.005914449691772461 nb_pixel_total : 2233 time to create 1 rle with old method : 0.0024442672729492188 time for calcul the mask position with numpy : 0.006066322326660156 nb_pixel_total : 985 time to create 1 rle with old method : 0.0010902881622314453 time for calcul the mask position with numpy : 0.0061228275299072266 nb_pixel_total : 111 time to create 1 rle with old method : 0.00013971328735351562 time for calcul the mask position with numpy : 0.006039142608642578 nb_pixel_total : 3775 time to create 1 rle with old method : 0.004160881042480469 time for calcul the mask position with numpy : 0.006138324737548828 nb_pixel_total : 495 time to create 1 rle with old method : 0.000545501708984375 time for calcul the mask position with numpy : 0.005921125411987305 nb_pixel_total : 5422 time to create 1 rle with old method : 0.005555391311645508 time for calcul the mask position with numpy : 0.00585484504699707 nb_pixel_total : 313 time to create 1 rle with old method : 0.00035762786865234375 time for calcul the mask position with numpy : 0.006507396697998047 nb_pixel_total : 440 time to create 1 rle with old method : 0.0005166530609130859 time for calcul the mask position with numpy : 0.00671839714050293 nb_pixel_total : 153818 time to create 1 rle with new method : 0.33051490783691406 time for calcul the mask position with numpy : 0.006142139434814453 nb_pixel_total : 243 time to create 1 rle with old method : 0.00032782554626464844 time for calcul the mask position with numpy : 0.006155490875244141 nb_pixel_total : 742 time to create 1 rle with old method : 0.0008676052093505859 time for calcul the mask position with numpy : 0.0061299800872802734 nb_pixel_total : 165 time to create 1 rle with old method : 0.0002033710479736328 time for calcul the mask position with numpy : 0.0059452056884765625 nb_pixel_total : 659 time to create 1 rle with old method : 0.0007216930389404297 time for calcul the mask position with numpy : 0.0060880184173583984 nb_pixel_total : 459 time to create 1 rle with old method : 0.0005390644073486328 time for calcul the mask position with numpy : 0.006430864334106445 nb_pixel_total : 68 time to create 1 rle with old method : 0.00010609626770019531 time for calcul the mask position with numpy : 0.006060600280761719 nb_pixel_total : 801 time to create 1 rle with old method : 0.0008769035339355469 time for calcul the mask position with numpy : 0.0059359073638916016 nb_pixel_total : 1396 time to create 1 rle with old method : 0.001489877700805664 time for calcul the mask position with numpy : 0.006005525588989258 nb_pixel_total : 819 time to create 1 rle with old method : 0.0008571147918701172 time for calcul the mask position with numpy : 0.006365299224853516 nb_pixel_total : 216 time to create 1 rle with old method : 0.0003886222839355469 time for calcul the mask position with numpy : 0.00594329833984375 nb_pixel_total : 12524 time to create 1 rle with old method : 0.013307332992553711 time for calcul the mask position with numpy : 0.0059545040130615234 nb_pixel_total : 262 time to create 1 rle with old method : 0.00030517578125 time for calcul the mask position with numpy : 0.0059969425201416016 nb_pixel_total : 9458 time to create 1 rle with old method : 0.010198116302490234 time for calcul the mask position with numpy : 0.006219148635864258 nb_pixel_total : 578 time to create 1 rle with old method : 0.0006606578826904297 time for calcul the mask position with numpy : 0.007321834564208984 nb_pixel_total : 99 time to create 1 rle with old method : 0.00014495849609375 time for calcul the mask position with numpy : 0.00677943229675293 nb_pixel_total : 726 time to create 1 rle with old method : 0.0008370876312255859 time for calcul the mask position with numpy : 0.006683826446533203 nb_pixel_total : 1465 time to create 1 rle with old method : 0.0016829967498779297 time for calcul the mask position with numpy : 0.007725238800048828 nb_pixel_total : 787 time to create 1 rle with old method : 0.0009188652038574219 time for calcul the mask position with numpy : 0.0071718692779541016 nb_pixel_total : 4821 time to create 1 rle with old method : 0.005033254623413086 time for calcul the mask position with numpy : 0.0059719085693359375 nb_pixel_total : 87 time to create 1 rle with old method : 0.00011539459228515625 time for calcul the mask position with numpy : 0.006089210510253906 nb_pixel_total : 1672 time to create 1 rle with old method : 0.00186920166015625 time for calcul the mask position with numpy : 0.0060846805572509766 nb_pixel_total : 410 time to create 1 rle with old method : 0.0004639625549316406 time for calcul the mask position with numpy : 0.0061070919036865234 nb_pixel_total : 218 time to create 1 rle with old method : 0.00026702880859375 time for calcul the mask position with numpy : 0.006170988082885742 nb_pixel_total : 244 time to create 1 rle with old method : 0.00028824806213378906 time for calcul the mask position with numpy : 0.0060808658599853516 nb_pixel_total : 1499 time to create 1 rle with old method : 0.0015788078308105469 time for calcul the mask position with numpy : 0.005957365036010742 nb_pixel_total : 43 time to create 1 rle with old method : 0.00019097328186035156 time for calcul the mask position with numpy : 0.0059778690338134766 nb_pixel_total : 1246 time to create 1 rle with old method : 0.0013861656188964844 time for calcul the mask position with numpy : 0.005902290344238281 nb_pixel_total : 22 time to create 1 rle with old method : 8.344650268554688e-05 time for calcul the mask position with numpy : 0.0059795379638671875 nb_pixel_total : 393 time to create 1 rle with old method : 0.0004487037658691406 time for calcul the mask position with numpy : 0.006138801574707031 nb_pixel_total : 214 time to create 1 rle with old method : 0.0002543926239013672 time for calcul the mask position with numpy : 0.006049156188964844 nb_pixel_total : 1522 time to create 1 rle with old method : 0.0016238689422607422 time for calcul the mask position with numpy : 0.0071256160736083984 nb_pixel_total : 43 time to create 1 rle with old method : 9.822845458984375e-05 time for calcul the mask position with numpy : 0.006662607192993164 nb_pixel_total : 452 time to create 1 rle with old method : 0.0005245208740234375 time for calcul the mask position with numpy : 0.006881237030029297 nb_pixel_total : 91 time to create 1 rle with old method : 0.00011944770812988281 time for calcul the mask position with numpy : 0.008376598358154297 nb_pixel_total : 1661 time to create 1 rle with old method : 0.0018877983093261719 time for calcul the mask position with numpy : 0.007058858871459961 nb_pixel_total : 368 time to create 1 rle with old method : 0.0004296302795410156 time for calcul the mask position with numpy : 0.0062100887298583984 nb_pixel_total : 407 time to create 1 rle with old method : 0.0004715919494628906 time for calcul the mask position with numpy : 0.006125688552856445 nb_pixel_total : 105 time to create 1 rle with old method : 0.00013327598571777344 time for calcul the mask position with numpy : 0.00619959831237793 nb_pixel_total : 216 time to create 1 rle with old method : 0.0002589225769042969 time for calcul the mask position with numpy : 0.006711721420288086 nb_pixel_total : 106010 time to create 1 rle with old method : 0.11187219619750977 time for calcul the mask position with numpy : 0.007819652557373047 nb_pixel_total : 338 time to create 1 rle with old method : 0.00042724609375 time for calcul the mask position with numpy : 0.00709223747253418 nb_pixel_total : 75 time to create 1 rle with old method : 0.00010776519775390625 time for calcul the mask position with numpy : 0.00628352165222168 nb_pixel_total : 4 time to create 1 rle with old method : 2.6702880859375e-05 time for calcul the mask position with numpy : 0.006425619125366211 nb_pixel_total : 44126 time to create 1 rle with old method : 0.04613351821899414 time for calcul the mask position with numpy : 0.006056547164916992 nb_pixel_total : 500 time to create 1 rle with old method : 0.0007228851318359375 time for calcul the mask position with numpy : 0.006124734878540039 nb_pixel_total : 290 time to create 1 rle with old method : 0.0003459453582763672 time for calcul the mask position with numpy : 0.006409406661987305 nb_pixel_total : 272 time to create 1 rle with old method : 0.0003247261047363281 time for calcul the mask position with numpy : 0.0063555240631103516 nb_pixel_total : 916 time to create 1 rle with old method : 0.0010223388671875 time for calcul the mask position with numpy : 0.006556987762451172 nb_pixel_total : 2749 time to create 1 rle with old method : 0.0030069351196289062 time for calcul the mask position with numpy : 0.00640106201171875 nb_pixel_total : 217 time to create 1 rle with old method : 0.00025963783264160156 time for calcul the mask position with numpy : 0.0062143802642822266 nb_pixel_total : 6 time to create 1 rle with old method : 2.5272369384765625e-05 time for calcul the mask position with numpy : 0.006216526031494141 nb_pixel_total : 109 time to create 1 rle with old method : 0.00013184547424316406 time for calcul the mask position with numpy : 0.006226778030395508 nb_pixel_total : 96 time to create 1 rle with old method : 0.00012683868408203125 time for calcul the mask position with numpy : 0.006215095520019531 nb_pixel_total : 1690 time to create 1 rle with old method : 0.0018320083618164062 time for calcul the mask position with numpy : 0.0061376094818115234 nb_pixel_total : 158 time to create 1 rle with old method : 0.00018286705017089844 time for calcul the mask position with numpy : 0.00616002082824707 nb_pixel_total : 54 time to create 1 rle with old method : 8.320808410644531e-05 time for calcul the mask position with numpy : 0.006212711334228516 nb_pixel_total : 73 time to create 1 rle with old method : 9.322166442871094e-05 time for calcul the mask position with numpy : 0.0061342716217041016 nb_pixel_total : 607 time to create 1 rle with old method : 0.0006909370422363281 time for calcul the mask position with numpy : 0.008504152297973633 nb_pixel_total : 117 time to create 1 rle with old method : 0.0001571178436279297 time for calcul the mask position with numpy : 0.008449792861938477 nb_pixel_total : 39 time to create 1 rle with old method : 5.3882598876953125e-05 time for calcul the mask position with numpy : 0.008254051208496094 nb_pixel_total : 3 time to create 1 rle with old method : 2.9325485229492188e-05 time for calcul the mask position with numpy : 0.008588790893554688 nb_pixel_total : 19 time to create 1 rle with old method : 8.511543273925781e-05 time for calcul the mask position with numpy : 0.008577585220336914 nb_pixel_total : 996 time to create 1 rle with old method : 0.001110076904296875 time for calcul the mask position with numpy : 0.00868082046508789 nb_pixel_total : 2130 time to create 1 rle with old method : 0.0023245811462402344 time for calcul the mask position with numpy : 0.008574724197387695 nb_pixel_total : 226 time to create 1 rle with old method : 0.00026702880859375 create new chi : 1.795107364654541 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.0012159347534179688 batch 1 Loaded 85 chid ids of type : 4230 Number RLEs to save : 9535 TO DO : save crop sub photo not yet done ! save time : 0.7445721626281738 map_output_result : {1333956514: (0.0, 'Should be the crop_list due to order', 0.0), 1333956513: (0.0, 'Should be the crop_list due to order', 0.0), 1333956512: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1333956514, 1333956513, 1333956512] Looping around the photos to save general results len do output : 3 /1333956514.Didn't retrieve data . /1333956513.Didn't retrieve data . /1333956512.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2538723') ('4234', '20162692', '1333956514', None, None, None, None, None, '2538723') ('4234', None, None, None, None, None, None, None, '2538723') ('4234', '20162692', '1333956513', None, None, None, None, None, '2538723') ('4234', None, None, None, None, None, None, None, '2538723') ('4234', '20162692', '1333956512', None, None, None, None, None, '2538723') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 1.2207884788513184 save_final save missing photos in datou_result : time spend for datou_step_exec : 16.562791109085083 time spend to save output : 1.221144437789917 total time spend for step 4 : 17.783935546875 step5:crop_condition Mon Feb 3 01:21:19 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4230 Loading chi in step crop for list_pids : 3 ! batch 1 Loaded 240 chid ids of type : 4230 begin to crop the class : papier param for this class : {'min_score': 0.6} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 61 About to insert : list_path_to_insert length 61 new photo from crops ! About to upload 61 photos upload in portfolio : 4869462 init cache_photo without model_param we have 61 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738542081_887412 we have uploaded 61 photos in the portfolio 4869462 time of upload the photos Elapsed time : 13.470701694488525 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.6} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 32 About to insert : list_path_to_insert length 32 new photo from crops ! About to upload 32 photos upload in portfolio : 4869462 init cache_photo without model_param we have 32 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738542095_887412 we have uploaded 32 photos in the portfolio 4869462 time of upload the photos Elapsed time : 8.56833553314209 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.6} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.6} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 15 About to insert : list_path_to_insert length 15 new photo from crops ! About to upload 15 photos upload in portfolio : 4869462 init cache_photo without model_param we have 15 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738542104_887412 we have uploaded 15 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.8945510387420654 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.6} filtre for class : autre hashtag_id of this class : 494826614 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 33 About to insert : list_path_to_insert length 33 new photo from crops ! About to upload 33 photos upload in portfolio : 4869462 init cache_photo without model_param we have 33 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738542108_887412 we have uploaded 33 photos in the portfolio 4869462 time of upload the photos Elapsed time : 7.165703296661377 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.6} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.6} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1333956514, 1333956513, 1333956512] Looping around the photos to save general results len do output : 141 /1334042119Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042120Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042121Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042122Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042123Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042124Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042125Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042126Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042127Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042128Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042129Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042130Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042131Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042132Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042133Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042134Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042135Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042136Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042137Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042138Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042140Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042141Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042142Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042143Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042144Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042145Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042146Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042147Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042148Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042149Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042150Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042151Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042152Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042153Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042154Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042155Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042156Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042157Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042158Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042159Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042160Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042161Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042162Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042163Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042164Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042165Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042166Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042167Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042168Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042169Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042170Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042171Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042172Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042173Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042174Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042175Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042176Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042177Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042178Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042179Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042180Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042183Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042184Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042185Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042186Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042188Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042190Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042191Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042193Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042194Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042195Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042196Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042197Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042198Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042199Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042200Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042201Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042202Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042203Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042204Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042205Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042206Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042207Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042208Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042209Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042210Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042211Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042213Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042214Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042215Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042216Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042217Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042218Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042220Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042221Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042222Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042223Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042224Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042225Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042226Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042227Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042228Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042229Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042230Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042231Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042232Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042233Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042234Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042235Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042236Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042237Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042238Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042239Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042240Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042241Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042242Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042243Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042244Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042245Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042246Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042247Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042248Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042249Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042250Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042251Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042252Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042253Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042254Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042255Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042256Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042257Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042258Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042259Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042260Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042261Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042262Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042263Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042264Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042265Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042266Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334042267Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2538723') ('4234', '20162692', '1333956514', None, None, None, None, None, '2538723') ('4234', None, None, None, None, None, None, None, '2538723') ('4234', '20162692', '1333956513', None, None, None, None, None, '2538723') ('4234', None, None, None, None, None, None, None, '2538723') ('4234', '20162692', '1333956512', None, None, None, None, None, '2538723') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 426 time used for this insertion : 0.05496954917907715 save_final save missing photos in datou_result : time spend for datou_step_exec : 36.35727143287659 time spend to save output : 0.05925869941711426 total time spend for step 5 : 36.4165301322937 step6:thcl Mon Feb 3 01:21:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! we are using the classfication for only one thcl 3237 time to import caffe and check if the image exist : 0.0032384395599365234 time to convert the images to numpy array : 0.025457382202148438 time to import caffe and check if the image exist : 0.007366657257080078 time to convert the images to numpy array : 0.03809714317321777 time to import caffe and check if the image exist : 0.010805845260620117 time to convert the images to numpy array : 0.03867840766906738 time to import caffe and check if the image exist : 0.010459423065185547 time to convert the images to numpy array : 0.04955744743347168 time to import caffe and check if the image exist : 0.009076595306396484 time to convert the images to numpy array : 0.05114293098449707 time to import caffe and check if the image exist : 0.012864351272583008 time to convert the images to numpy array : 0.05136227607727051 time to import caffe and check if the image exist : 0.014486551284790039 time to convert the images to numpy array : 0.05060100555419922 time to import caffe and check if the image exist : 0.01528620719909668 time to convert the images to numpy array : 0.04990053176879883 time to import caffe and check if the image exist : 0.026620149612426758 time to convert the images to numpy array : 0.03869962692260742 time to import caffe and check if the image exist : 0.0216367244720459 time to convert the images to numpy array : 0.044016122817993164 total time to convert the images to numpy array : 0.28836750984191895 list photo_ids error: [] list photo_ids correct : [1334042262, 1334042263, 1334042264, 1334042265, 1334042266, 1334042267, 1334042180, 1334042183, 1334042184, 1334042185, 1334042186, 1334042188, 1334042190, 1334042191, 1334042193, 1334042194, 1334042195, 1334042196, 1334042197, 1334042198, 1334042199, 1334042150, 1334042151, 1334042152, 1334042153, 1334042154, 1334042155, 1334042156, 1334042157, 1334042158, 1334042159, 1334042160, 1334042161, 1334042162, 1334042163, 1334042164, 1334042134, 1334042135, 1334042136, 1334042137, 1334042138, 1334042140, 1334042141, 1334042142, 1334042143, 1334042144, 1334042145, 1334042146, 1334042147, 1334042148, 1334042149, 1334042165, 1334042166, 1334042167, 1334042168, 1334042169, 1334042170, 1334042171, 1334042172, 1334042173, 1334042174, 1334042175, 1334042176, 1334042177, 1334042178, 1334042179, 1334042119, 1334042120, 1334042121, 1334042122, 1334042123, 1334042124, 1334042125, 1334042126, 1334042127, 1334042128, 1334042129, 1334042130, 1334042131, 1334042132, 1334042133, 1334042200, 1334042201, 1334042202, 1334042203, 1334042204, 1334042205, 1334042206, 1334042207, 1334042208, 1334042209, 1334042210, 1334042211, 1334042213, 1334042214, 1334042215, 1334042216, 1334042217, 1334042218, 1334042220, 1334042221, 1334042222, 1334042223, 1334042224, 1334042225, 1334042226, 1334042227, 1334042228, 1334042229, 1334042230, 1334042231, 1334042247, 1334042248, 1334042249, 1334042250, 1334042251, 1334042252, 1334042253, 1334042254, 1334042255, 1334042256, 1334042257, 1334042258, 1334042259, 1334042260, 1334042261, 1334042232, 1334042233, 1334042234, 1334042235, 1334042236, 1334042237, 1334042238, 1334042239, 1334042240, 1334042241, 1334042242, 1334042243, 1334042244, 1334042245, 1334042246] number of photos to traite : 141 try to delete the photos incorrect in DB tagging for thcl : 3237 To do loadFromThcl(), then load ParamDescType : thcl3237 thcls : [{'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'}] thcl {'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'} Update svm_hashtag_type_desc : 5561 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) To loadFromThcl() : net_5561 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 5706 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_rubbia_refus_2500 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_rubbia_refus_2500 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_rubbia_refus_2500 /data/models_weight/learn_rubbia_refus_2500/caffemodel size_local : 94358479 size in s3 : 94358479 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:16 caffemodel already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt size_local : 32544 size in s3 : 32544 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:15 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:09:52 mean.npy already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/synset_words.txt size_local : 334 size in s3 : 334 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:10:06 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt caffemodel_filename : /data/models_weight/learn_rubbia_refus_2500/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 5487 max_wait_temp : 1 max_wait : 0 dict_keys(['prob', 'pool5']) time used to do the prepocess of the images : 0.6410095691680908 time used to do the prediction : 1.1832118034362793 save descriptor for thcl : 3237 time to traite the descriptors : 0.8211669921875 Catched exception ! Connect or reconnect ! storage_type for insertDescriptorsMulti : 3 To insert : 1334042262 To insert : 1334042263 To insert : 1334042264 To insert : 1334042265 To insert : 1334042266 To insert : 1334042267 To insert : 1334042180 To insert : 1334042183 To insert : 1334042184 To insert : 1334042185 To insert : 1334042186 To insert : 1334042188 To insert : 1334042190 To insert : 1334042191 To insert : 1334042193 To insert : 1334042194 To insert : 1334042195 To insert : 1334042196 To insert : 1334042197 To insert : 1334042198 To insert : 1334042199 To insert : 1334042150 To insert : 1334042151 To insert : 1334042152 To insert : 1334042153 To insert : 1334042154 To insert : 1334042155 To insert : 1334042156 To insert : 1334042157 To insert : 1334042158 To insert : 1334042159 To insert : 1334042160 To insert : 1334042161 To insert : 1334042162 To insert : 1334042163 To insert : 1334042164 To insert : 1334042134 To insert : 1334042135 To insert : 1334042136 To insert : 1334042137 To insert : 1334042138 To insert : 1334042140 To insert : 1334042141 To insert : 1334042142 To insert : 1334042143 To insert : 1334042144 To insert : 1334042145 To insert : 1334042146 To insert : 1334042147 To insert : 1334042148 To insert : 1334042149 To insert : 1334042165 To insert : 1334042166 To insert : 1334042167 To insert : 1334042168 To insert : 1334042169 To insert : 1334042170 To insert : 1334042171 To insert : 1334042172 To insert : 1334042173 To insert : 1334042174 To insert : 1334042175 To insert : 1334042176 To insert : 1334042177 To insert : 1334042178 To insert : 1334042179 To insert : 1334042119 To insert : 1334042120 To insert : 1334042121 To insert : 1334042122 To insert : 1334042123 To insert : 1334042124 To insert : 1334042125 To insert : 1334042126 To insert : 1334042127 To insert : 1334042128 To insert : 1334042129 To insert : 1334042130 To insert : 1334042131 To insert : 1334042132 To insert : 1334042133 To insert : 1334042200 To insert : 1334042201 To insert : 1334042202 To insert : 1334042203 To insert : 1334042204 To insert : 1334042205 To insert : 1334042206 To insert : 1334042207 To insert : 1334042208 To insert : 1334042209 To insert : 1334042210 To insert : 1334042211 To insert : 1334042213 To insert : 1334042214 To insert : 1334042215 To insert : 1334042216 To insert : 1334042217 To insert : 1334042218 To insert : 1334042220 To insert : 1334042221 To insert : 1334042222 To insert : 1334042223 To insert : 1334042224 To insert : 1334042225 To insert : 1334042226 To insert : 1334042227 To insert : 1334042228 To insert : 1334042229 To insert : 1334042230 To insert : 1334042231 To insert : 1334042247 To insert : 1334042248 To insert : 1334042249 To insert : 1334042250 To insert : 1334042251 To insert : 1334042252 To insert : 1334042253 To insert : 1334042254 To insert : 1334042255 To insert : 1334042256 To insert : 1334042257 To insert : 1334042258 To insert : 1334042259 To insert : 1334042260 To insert : 1334042261 To insert : 1334042232 To insert : 1334042233 To insert : 1334042234 To insert : 1334042235 To insert : 1334042236 To insert : 1334042237 To insert : 1334042238 To insert : 1334042239 To insert : 1334042240 To insert : 1334042241 To insert : 1334042242 To insert : 1334042243 To insert : 1334042244 To insert : 1334042245 To insert : 1334042246 time to insert the descriptors : 25.384891748428345 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos SAVE THCL : begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 220 time used for this insertion : 0.020312786102294922 save missing photos in datou_result : time spend for datou_step_exec : 32.14283108711243 time spend to save output : 0.07077288627624512 total time spend for step 6 : 32.21360397338867 step7:ventilate_hashtags_in_portfolio Mon Feb 3 01:22:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 20162692 get user id for portfolio 20162692 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20162692 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('PET_clair','Papier','PET_fonce','Tetrapak','autre','carton','environnement','PEHD','Carton','pehd','metal','pet_fonce','mal_croppe','flou','papier','pet_clair','refus','Film_plastique')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20162692 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('PET_clair','Papier','PET_fonce','Tetrapak','autre','carton','environnement','PEHD','Carton','pehd','metal','pet_fonce','mal_croppe','flou','papier','pet_clair','refus','Film_plastique')) AND mptpi.`min_score`=0.6 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20162692 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('PET_clair','Papier','PET_fonce','Tetrapak','autre','carton','environnement','PEHD','Carton','pehd','metal','pet_fonce','mal_croppe','flou','papier','pet_clair','refus','Film_plastique')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20162692 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('PET_clair','Papier','PET_fonce','Tetrapak','autre','carton','environnement','PEHD','Carton','pehd','metal','pet_fonce','mal_croppe','flou','papier','pet_clair','refus','Film_plastique')) AND mptpi.`min_score`=0.6 To do lien utilise dans velours : https://www.fotonower.com/velours/20163895,20163894,20163891,20163883,20163884,20163888,20163886,20163889,20163890,20163892,20163893,20163896,20163897?tags=pet_clair,papier,pet_fonce,tetrapak,autre,carton,environnement,pehd,metal,mal_croppe,flou,refus,film_plastique&datou_id_consolidate=4235&port_consolidate=20162692 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1333956514, 1333956513, 1333956512] Looping around the photos to save general results len do output : 1 /20162692. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2538723') ('4234', '20162692', '1333956514', None, None, None, None, None, '2538723') ('4234', None, None, None, None, None, None, None, '2538723') ('4234', '20162692', '1333956513', None, None, None, None, None, '2538723') ('4234', None, None, None, None, None, None, None, '2538723') ('4234', '20162692', '1333956512', None, None, None, None, None, '2538723') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 4 time used for this insertion : 0.014697790145874023 save_final save missing photos in datou_result : time spend for datou_step_exec : 3.6715471744537354 time spend to save output : 0.01500391960144043 total time spend for step 7 : 3.686551094055176 step8:final Mon Feb 3 01:22:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1333956514: ('0.2004456322145325',), 1333956513: ('0.2004456322145325',), 1333956512: ('0.2004456322145325',)} new output for save of step final : {1333956514: ('0.2004456322145325',), 1333956513: ('0.2004456322145325',), 1333956512: ('0.2004456322145325',)} [1333956514, 1333956513, 1333956512] Looping around the photos to save general results len do output : 3 /1333956514.Didn't retrieve data . /1333956513.Didn't retrieve data . /1333956512.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2538723') ('4234', '20162692', '1333956514', None, None, None, None, None, '2538723') ('4234', None, None, None, None, None, None, None, '2538723') ('4234', '20162692', '1333956513', None, None, None, None, None, '2538723') ('4234', None, None, None, None, None, None, None, '2538723') ('4234', '20162692', '1333956512', None, None, None, None, None, '2538723') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.011764287948608398 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.12281656265258789 time spend to save output : 0.012004613876342773 total time spend for step 8 : 0.13482117652893066 step9:velours_tree Mon Feb 3 01:22:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure list_portfolios : 20163895,20163894,20163891,20163883,20163884,20163888,20163886,20163889,20163890,20163892,20163893,20163896,20163897 photo desc type : 5561 - Retrieving photos to tag... query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20163895 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20163894 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20163891 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20163883 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20163884 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20163888 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20163886 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20163889 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20163890 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20163892 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20163893 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20163896 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20163897 ORDER BY ph.size desc - Loading descriptors... Size : 2048 len(descriptors) : 141 Compute structured hierarchical clustering... ward : AgglomerativeClustering(n_clusters=6) ward.labels_ : [3 0 0 1 1 0 0 0 2 4 0 2 0 2 2 2 1 0 0 0 1 5 0 0 1 1 0 0 0 0 4 0 0 0 2 2 2 0 0 1 0 0 0 1 0 1 0 0 0 4 0 5 0 2 2 2 0 0 1 0 0 5 3 2 1 0 2 2 2 0 0 0 0 5 3 2 0 2 2 2 0 0 2 0 5 3 2 0 0 2 2 0 0 1 1 0 2 2 2 1 0 2 1 0 2 1 0 2 3 0 2 0 2 2 0 0 2 2 0 0 2 2 3 2 0 0 0 2 5 0 2 5 3 3 2 0 2 5 2 0 2] Elapsed time: 0.019112348556518555 graph_id used : 75563 - Beta version, working pretty good on 11-5-16 ! https://www.fotonower.com/velours/20163895,20163894,20163891,20163883,20163884,20163888,20163886,20163889,20163890,20163892,20163893,20163896,20163897?tags=pet_clair,papier,pet_fonce,tetrapak,autre,carton,environnement,pehd,metal,mal_croppe,flou,refus,film_plastique&datou_id_consolidate=4235&port_consolidate=20162692&tree_id=75563 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : velours_tree we use saveGeneral [1333956514, 1333956513, 1333956512] Looping around the photos to save general results len do output : 1 /20162692Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2538723') ('4234', '20162692', '1333956514', None, None, None, None, None, '2538723') ('4234', None, None, None, None, None, None, None, '2538723') ('4234', '20162692', '1333956513', None, None, None, None, None, '2538723') ('4234', None, None, None, None, None, None, None, '2538723') ('4234', '20162692', '1333956512', None, None, None, None, None, '2538723') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 4 time used for this insertion : 0.012146711349487305 save_final save missing photos in datou_result : time spend for datou_step_exec : 10.823641777038574 time spend to save output : 0.012403488159179688 total time spend for step 9 : 10.836045265197754 step10:send_mail_cod Mon Feb 3 01:22:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Qualipapia_P20162692_03-02-2025_01_22_42.pdf 20163877 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette201638771738542162 20163876 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette201638761738542163 20163873 imagette201638731738542164 20163865 imagette201638651738542164 20163866 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette201638661738542164 20163870 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette201638701738542165 20163871 imagette201638711738542167 20163872 imagette201638721738542167 20163874 imagette201638741738542167 20163875 imagette201638751738542167 20163878 imagette201638781738542167 20163879 imagette201638791738542167 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20162692 and hashtag_type = 4230 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20163895,20163894,20163891,20163883,20163884,20163888,20163886,20163889,20163890,20163892,20163893,20163896,20163897?tags=pet_clair,papier,pet_fonce,tetrapak,autre,carton,environnement,pehd,metal,mal_croppe,flou,refus,film_plastique&datou_id_consolidate=4235&port_consolidate=20162692&tree_id=75563 your option no_mail is active, we will not send the real mail to your client args[1333956514] : ((1333956514, -7.645552774554034, 492609224), (1333956514, -0.052413409457380634, 2107752395), '0.2004456322145325') We are sending mail with results at cod@fotonower.com args[1333956513] : ((1333956513, -7.689588777369138, 492609224), (1333956513, -0.02890676387175458, 2107752395), '0.2004456322145325') We are sending mail with results at cod@fotonower.com args[1333956512] : ((1333956512, -7.709285482234909, 492609224), (1333956512, 0.0117537339365441, 2107752395), '0.2004456322145325') We are sending mail with results at cod@fotonower.com refus_total : 0.2004456322145325 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20162692 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20162692_03-02-2025_01_22_42.pdf results_Qualipapia_P20162692_03-02-2025_01_22_42.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20162692_03-02-2025_01_22_42.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4234','20162692','results_Qualipapia_P20162692_03-02-2025_01_22_42.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20162692_03-02-2025_01_22_42.pdf','pdf','','0.16','0.2004456322145325') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1333956514, 1333956513, 1333956512] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2538723') ('4234', '20162692', '1333956514', None, None, None, None, None, '2538723') ('4234', None, None, None, None, None, None, None, '2538723') ('4234', '20162692', '1333956513', None, None, None, None, None, '2538723') ('4234', None, None, None, None, None, None, None, '2538723') ('4234', '20162692', '1333956512', None, None, None, None, None, '2538723') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.013055562973022461 save_final save missing photos in datou_result : time spend for datou_step_exec : 6.33936619758606 time spend to save output : 0.013211250305175781 total time spend for step 10 : 6.352577447891235 step11:split_time_score Mon Feb 3 01:22:49 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 3442, 'mtr_user_id': 31, 'name': 'classifieur_2camions_valcor_021122_v1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'deux_camions,camion_droite,camion_gauche,pas_de_camion', 'svm_portfolios_learning': '7659379,7659034,7657685,7657114', 'photo_hashtag_type': 4458, 'photo_desc_type': 5723, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107760533,2107760534,2107760535,2107760536'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('15', 3),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 02022025 20162692 Nombre de photos uploadées : 3 / 23040 (0%) 02022025 20162692 Nombre de photos taguées (types de déchets): 3 / 3 (100%) 02022025 20162692 Nombre de photos taguées (volume) : 0 / 3 (0%) elapsed_time : load_data_split_time_score 1.6689300537109375e-06 elapsed_time : order_list_meta_photo_and_scores 5.4836273193359375e-06 ??? elapsed_time : fill_and_build_computed_from_old_data 0.00025010108947753906 elapsed_time : insert_dashboard_record_day_entry 0.02312183380126953 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162644 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162662 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162664 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162665 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162666 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162667 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162668 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162669 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162670 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162671 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162672 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162673 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162674 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162675 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162676 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162677 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162678 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162679 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162680 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162681 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162682 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162683 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162684 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162685 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162686 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162687 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162688 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162689 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162690 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162691 order by id desc limit 1 Qualite : 0.2004456322145325 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20162692_03-02-2025_01_22_42.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162692 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20162692 AND mptpi.`type`=4230 To do Qualite : 0.20880203185509494 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20162693_03-02-2025_01_20_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162693 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20162693 AND mptpi.`type`=4230 To do Qualite : 0.18514600417968427 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20162694_03-02-2025_01_16_33.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162694 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20162694 AND mptpi.`type`=4230 To do Qualite : 0.17908366031743364 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20162695_03-02-2025_00_51_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162695 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20162695 AND mptpi.`type`=4230 To do Qualite : 0.19305288886737082 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20162696_03-02-2025_00_44_24.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162696 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20162696 AND mptpi.`type`=4230 To do Qualite : 0.19328242832005024 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20162697_03-02-2025_00_46_39.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162697 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20162697 AND mptpi.`type`=4230 To do Qualite : 0.17139189028903767 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20162698_03-02-2025_00_34_38.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162698 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20162698 AND mptpi.`type`=4230 To do Qualite : 0.1891712268968112 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20162699_03-02-2025_00_31_41.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162699 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20162699 AND mptpi.`type`=4230 To do Qualite : 0.17923304163593254 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20162700_03-02-2025_00_27_05.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162700 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20162700 AND mptpi.`type`=4230 To do Qualite : 0.21157504534527558 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20162701_03-02-2025_00_24_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162701 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20162701 AND mptpi.`type`=4230 To do Qualite : 0.19050968855918649 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20162702_03-02-2025_00_18_27.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162702 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20162702 AND mptpi.`type`=4230 To do Qualite : 0.21383786583408712 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20162703_03-02-2025_00_16_36.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162703 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20162703 AND mptpi.`type`=4230 To do Qualite : 0.16928211545032903 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20162704_03-02-2025_00_10_23.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20162704 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20162704 AND mptpi.`type`=4230 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'02022025': {'nb_upload': 3, 'nb_taggue_class': 3, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1333956514, 1333956513, 1333956512] Looping around the photos to save general results len do output : 1 /20162692Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2538723') ('4234', '20162692', '1333956514', None, None, None, None, None, '2538723') ('4234', None, None, None, None, None, None, None, '2538723') ('4234', '20162692', '1333956513', None, None, None, None, None, '2538723') ('4234', None, None, None, None, None, None, None, '2538723') ('4234', '20162692', '1333956512', None, None, None, None, None, '2538723') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 4 time used for this insertion : 0.014539003372192383 save_final save missing photos in datou_result : time spend for datou_step_exec : 25.508599758148193 time spend to save output : 0.014737844467163086 total time spend for step 11 : 25.523337602615356 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 1 len(input) + len(total_photo_id_missing) : 3 set_done_treatment 37.04user 28.68system 2:50.52elapsed 38%CPU (0avgtext+0avgdata 2498132maxresident)k 1144inputs+12664outputs (5major+2068792minor)pagefaults 0swaps