python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 4234 -l 5 ' -s datou_current_4234 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3622369 load datou : 4234 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load THCL from format json or kwargs add thcl : 3237 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig add pdt : 5561 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4234, datou_cur_ids : ['2634220'] with mtr_portfolio_ids : ['21027618'] and first list_photo_ids : [] new path : /proc/3622369/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, brightness, blur_detection, rle_unique_nms_with_priority, crop_condition, thcl, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , BFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 time to download the photos : 0.6480426788330078 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Wed Mar 5 06:20:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3185 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-03-05 06:20:32.405533: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-03-05 06:20:32.431124: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-03-05 06:20:32.433207: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fed00000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-03-05 06:20:32.433244: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-03-05 06:20:32.437373: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-03-05 06:20:32.602184: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x4fe1b60 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-03-05 06:20:32.602248: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-03-05 06:20:32.603424: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-05 06:20:32.603898: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-05 06:20:32.607205: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-05 06:20:32.609984: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-05 06:20:32.610484: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-05 06:20:32.613401: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-05 06:20:32.614811: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-05 06:20:32.620532: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-05 06:20:32.621664: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-05 06:20:32.621757: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-05 06:20:32.622332: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-05 06:20:32.622348: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-05 06:20:32.622358: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-05 06:20:32.623306: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2514 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-03-05 06:20:32.899598: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-05 06:20:32.899711: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-05 06:20:32.899736: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-05 06:20:32.899757: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-05 06:20:32.899777: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-05 06:20:32.899797: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-05 06:20:32.899817: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-05 06:20:32.899837: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-05 06:20:32.900843: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-05 06:20:32.902011: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-05 06:20:32.902050: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-05 06:20:32.902069: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-05 06:20:32.902087: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-05 06:20:32.902104: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-05 06:20:32.902121: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-05 06:20:32.902138: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-05 06:20:32.902156: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-05 06:20:32.903219: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-05 06:20:32.903266: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-05 06:20:32.903277: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-05 06:20:32.903286: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-05 06:20:32.904329: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2514 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-03-05 06:20:41.353283: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-05 06:20:41.556631: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-05 06:20:43.257115: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-05 06:20:43.257226: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-05 06:20:43.264359: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-05 06:20:43.264408: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-05 06:20:43.322020: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-05 06:20:43.322120: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-05 06:20:43.368325: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.09GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-05 06:20:43.368378: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.09GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-05 06:20:43.422166: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.15GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-05 06:20:43.422225: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.15GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-05 06:20:43.424654: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.46G (1564344320 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.425198: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.31G (1407909888 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.425751: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.18G (1267118848 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.425778: W tensorflow/core/common_runtime/bfc_allocator.cc:311] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature. 2025-03-05 06:20:43.448184: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.448936: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.479635: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.480274: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.481946: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.482513: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.489353: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.490025: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.491898: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.492489: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.499135: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.499772: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.501521: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.502087: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.530621: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.531222: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.531783: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.532341: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.536215: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.536851: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.553398: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.553998: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.554589: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.555174: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.569788: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.570721: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.571621: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.572540: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.577983: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.579054: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.584910: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.585827: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.599387: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.599994: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.604283: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.604906: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.605520: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.606088: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.606997: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.607557: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.617949: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.618509: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.619102: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.619634: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.620168: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.621062: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.636501: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.637063: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.718718: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.719338: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.735042: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.735749: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.736419: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.737081: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.741331: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.741879: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.742433: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.743248: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.744241: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.744261: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-03-05 06:20:43.754887: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.755828: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.765803: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.766629: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.767267: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.767825: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.768368: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-05 06:20:43.768899: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.96G (2101215232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 1 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 21.00000 max: 219.00000 molded_images shape: (1, 640, 640, 3) min: -95.54375 max: 104.62734 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 6.604194641113281e-05 nb_pixel_total : 75 time to create 1 rle with old method : 0.00020051002502441406 length of segment : 12 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -122.97734 max: 150.97500 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 12 time for calcul the mask position with numpy : 5.602836608886719e-05 nb_pixel_total : 151 time to create 1 rle with old method : 0.0003218650817871094 length of segment : 10 time for calcul the mask position with numpy : 4.7206878662109375e-05 nb_pixel_total : 406 time to create 1 rle with old method : 0.0006468296051025391 length of segment : 24 time for calcul the mask position with numpy : 3.910064697265625e-05 nb_pixel_total : 306 time to create 1 rle with old method : 0.0004706382751464844 length of segment : 21 time for calcul the mask position with numpy : 4.792213439941406e-05 nb_pixel_total : 429 time to create 1 rle with old method : 0.0006337165832519531 length of segment : 25 time for calcul the mask position with numpy : 4.267692565917969e-05 nb_pixel_total : 546 time to create 1 rle with old method : 0.0007412433624267578 length of segment : 28 time for calcul the mask position with numpy : 3.695487976074219e-05 nb_pixel_total : 378 time to create 1 rle with old method : 0.000637054443359375 length of segment : 16 time for calcul the mask position with numpy : 5.793571472167969e-05 nb_pixel_total : 1037 time to create 1 rle with old method : 0.0013916492462158203 length of segment : 54 time for calcul the mask position with numpy : 5.078315734863281e-05 nb_pixel_total : 761 time to create 1 rle with old method : 0.0010974407196044922 length of segment : 29 time for calcul the mask position with numpy : 3.457069396972656e-05 nb_pixel_total : 192 time to create 1 rle with old method : 0.00028204917907714844 length of segment : 18 time for calcul the mask position with numpy : 3.647804260253906e-05 nb_pixel_total : 187 time to create 1 rle with old method : 0.0002994537353515625 length of segment : 27 time for calcul the mask position with numpy : 5.435943603515625e-05 nb_pixel_total : 1097 time to create 1 rle with old method : 0.0014395713806152344 length of segment : 54 time for calcul the mask position with numpy : 4.0531158447265625e-05 nb_pixel_total : 179 time to create 1 rle with old method : 0.00035381317138671875 length of segment : 24 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.51250 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 18 time for calcul the mask position with numpy : 6.556510925292969e-05 nb_pixel_total : 513 time to create 1 rle with old method : 0.0006794929504394531 length of segment : 35 time for calcul the mask position with numpy : 5.316734313964844e-05 nb_pixel_total : 884 time to create 1 rle with old method : 0.001123189926147461 length of segment : 50 time for calcul the mask position with numpy : 4.744529724121094e-05 nb_pixel_total : 369 time to create 1 rle with old method : 0.0005705356597900391 length of segment : 21 time for calcul the mask position with numpy : 5.817413330078125e-05 nb_pixel_total : 969 time to create 1 rle with old method : 0.0013692378997802734 length of segment : 23 time for calcul the mask position with numpy : 4.792213439941406e-05 nb_pixel_total : 464 time to create 1 rle with old method : 0.0006253719329833984 length of segment : 26 time for calcul the mask position with numpy : 3.647804260253906e-05 nb_pixel_total : 108 time to create 1 rle with old method : 0.00018644332885742188 length of segment : 11 time for calcul the mask position with numpy : 6.413459777832031e-05 nb_pixel_total : 1487 time to create 1 rle with old method : 0.0020563602447509766 length of segment : 42 time for calcul the mask position with numpy : 5.316734313964844e-05 nb_pixel_total : 755 time to create 1 rle with old method : 0.0010433197021484375 length of segment : 27 time for calcul the mask position with numpy : 5.459785461425781e-05 nb_pixel_total : 843 time to create 1 rle with old method : 0.0010883808135986328 length of segment : 33 time for calcul the mask position with numpy : 4.458427429199219e-05 nb_pixel_total : 504 time to create 1 rle with old method : 0.0007021427154541016 length of segment : 32 time for calcul the mask position with numpy : 5.269050598144531e-05 nb_pixel_total : 1092 time to create 1 rle with old method : 0.0014505386352539062 length of segment : 43 time for calcul the mask position with numpy : 3.814697265625e-05 nb_pixel_total : 255 time to create 1 rle with old method : 0.00036907196044921875 length of segment : 21 time for calcul the mask position with numpy : 7.200241088867188e-05 nb_pixel_total : 2024 time to create 1 rle with old method : 0.002469778060913086 length of segment : 63 time for calcul the mask position with numpy : 5.245208740234375e-05 nb_pixel_total : 715 time to create 1 rle with old method : 0.000985860824584961 length of segment : 63 time for calcul the mask position with numpy : 3.910064697265625e-05 nb_pixel_total : 312 time to create 1 rle with old method : 0.00044417381286621094 length of segment : 19 time for calcul the mask position with numpy : 3.314018249511719e-05 nb_pixel_total : 307 time to create 1 rle with old method : 0.00046706199645996094 length of segment : 20 time for calcul the mask position with numpy : 5.53131103515625e-05 nb_pixel_total : 1124 time to create 1 rle with old method : 0.0014772415161132812 length of segment : 27 time for calcul the mask position with numpy : 3.4809112548828125e-05 nb_pixel_total : 332 time to create 1 rle with old method : 0.0005128383636474609 length of segment : 22 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -120.65703 max: 147.89687 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 14 time for calcul the mask position with numpy : 6.771087646484375e-05 nb_pixel_total : 981 time to create 1 rle with old method : 0.0015270709991455078 length of segment : 19 time for calcul the mask position with numpy : 3.719329833984375e-05 nb_pixel_total : 203 time to create 1 rle with old method : 0.0002944469451904297 length of segment : 17 time for calcul the mask position with numpy : 3.600120544433594e-05 nb_pixel_total : 243 time to create 1 rle with old method : 0.00039267539978027344 length of segment : 15 time for calcul the mask position with numpy : 5.841255187988281e-05 nb_pixel_total : 1656 time to create 1 rle with old method : 0.0021059513092041016 length of segment : 47 time for calcul the mask position with numpy : 8.845329284667969e-05 nb_pixel_total : 2378 time to create 1 rle with old method : 0.003004312515258789 length of segment : 59 time for calcul the mask position with numpy : 5.53131103515625e-05 nb_pixel_total : 412 time to create 1 rle with old method : 0.0006275177001953125 length of segment : 28 time for calcul the mask position with numpy : 4.029273986816406e-05 nb_pixel_total : 402 time to create 1 rle with old method : 0.0009176731109619141 length of segment : 22 time for calcul the mask position with numpy : 4.839897155761719e-05 nb_pixel_total : 56 time to create 1 rle with old method : 0.00014734268188476562 length of segment : 11 time for calcul the mask position with numpy : 4.220008850097656e-05 nb_pixel_total : 418 time to create 1 rle with old method : 0.0005905628204345703 length of segment : 41 time for calcul the mask position with numpy : 6.222724914550781e-05 nb_pixel_total : 726 time to create 1 rle with old method : 0.0013158321380615234 length of segment : 19 time for calcul the mask position with numpy : 8.130073547363281e-05 nb_pixel_total : 993 time to create 1 rle with old method : 0.0014166831970214844 length of segment : 49 time for calcul the mask position with numpy : 9.1552734375e-05 nb_pixel_total : 328 time to create 1 rle with old method : 0.000514984130859375 length of segment : 37 time for calcul the mask position with numpy : 6.556510925292969e-05 nb_pixel_total : 252 time to create 1 rle with old method : 0.00038909912109375 length of segment : 24 time for calcul the mask position with numpy : 4.982948303222656e-05 nb_pixel_total : 239 time to create 1 rle with old method : 0.0005006790161132812 length of segment : 31 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -120.45781 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 6.437301635742188e-05 nb_pixel_total : 434 time to create 1 rle with old method : 0.0008945465087890625 length of segment : 28 time for calcul the mask position with numpy : 5.793571472167969e-05 nb_pixel_total : 334 time to create 1 rle with old method : 0.0006341934204101562 length of segment : 36 time for calcul the mask position with numpy : 5.4836273193359375e-05 nb_pixel_total : 79 time to create 1 rle with old method : 0.0002765655517578125 length of segment : 24 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -113.72734 max: 138.98672 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0003178119659423828 nb_pixel_total : 14050 time to create 1 rle with old method : 0.01823568344116211 length of segment : 259 Processing 1 images image shape: (400, 160, 3) min: 35.00000 max: 164.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 37.37734 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -121.51641 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 5.745887756347656e-05 nb_pixel_total : 1036 time to create 1 rle with old method : 0.0013489723205566406 length of segment : 46 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 451 time to create 1 rle with old method : 0.0006692409515380859 length of segment : 31 time for calcul the mask position with numpy : 7.510185241699219e-05 nb_pixel_total : 2287 time to create 1 rle with old method : 0.0030329227447509766 length of segment : 42 time for calcul the mask position with numpy : 9.72747802734375e-05 nb_pixel_total : 3537 time to create 1 rle with old method : 0.004334926605224609 length of segment : 86 time for calcul the mask position with numpy : 0.0001862049102783203 nb_pixel_total : 5213 time to create 1 rle with old method : 0.006560087203979492 length of segment : 129 time for calcul the mask position with numpy : 0.0002617835998535156 nb_pixel_total : 4562 time to create 1 rle with old method : 0.006606340408325195 length of segment : 134 time for calcul the mask position with numpy : 0.000110626220703125 nb_pixel_total : 3741 time to create 1 rle with old method : 0.004758358001708984 length of segment : 98 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -121.91875 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 15 time for calcul the mask position with numpy : 0.00010013580322265625 nb_pixel_total : 2798 time to create 1 rle with old method : 0.003415346145629883 length of segment : 68 time for calcul the mask position with numpy : 4.9114227294921875e-05 nb_pixel_total : 614 time to create 1 rle with old method : 0.0008168220520019531 length of segment : 33 time for calcul the mask position with numpy : 0.00024008750915527344 nb_pixel_total : 8076 time to create 1 rle with old method : 0.013643026351928711 length of segment : 209 time for calcul the mask position with numpy : 5.888938903808594e-05 nb_pixel_total : 322 time to create 1 rle with old method : 0.00044155120849609375 length of segment : 31 time for calcul the mask position with numpy : 8.678436279296875e-05 nb_pixel_total : 2439 time to create 1 rle with old method : 0.002895832061767578 length of segment : 86 time for calcul the mask position with numpy : 9.393692016601562e-05 nb_pixel_total : 1262 time to create 1 rle with old method : 0.0016088485717773438 length of segment : 83 time for calcul the mask position with numpy : 4.744529724121094e-05 nb_pixel_total : 268 time to create 1 rle with old method : 0.0003581047058105469 length of segment : 47 time for calcul the mask position with numpy : 0.00028705596923828125 nb_pixel_total : 15059 time to create 1 rle with old method : 0.017589092254638672 length of segment : 143 time for calcul the mask position with numpy : 8.559226989746094e-05 nb_pixel_total : 1924 time to create 1 rle with old method : 0.0025818347930908203 length of segment : 48 time for calcul the mask position with numpy : 6.4849853515625e-05 nb_pixel_total : 1825 time to create 1 rle with old method : 0.002474069595336914 length of segment : 45 time for calcul the mask position with numpy : 7.367134094238281e-05 nb_pixel_total : 1542 time to create 1 rle with old method : 0.0021245479583740234 length of segment : 39 time for calcul the mask position with numpy : 5.7220458984375e-05 nb_pixel_total : 431 time to create 1 rle with old method : 0.0005550384521484375 length of segment : 72 time for calcul the mask position with numpy : 8.678436279296875e-05 nb_pixel_total : 1474 time to create 1 rle with old method : 0.0020685195922851562 length of segment : 45 time for calcul the mask position with numpy : 0.0003039836883544922 nb_pixel_total : 6575 time to create 1 rle with old method : 0.010007381439208984 length of segment : 187 time for calcul the mask position with numpy : 9.679794311523438e-05 nb_pixel_total : 738 time to create 1 rle with old method : 0.0010118484497070312 length of segment : 45 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -121.11016 max: 150.94766 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 6.67572021484375e-05 nb_pixel_total : 501 time to create 1 rle with old method : 0.0007491111755371094 length of segment : 24 time for calcul the mask position with numpy : 8.487701416015625e-05 nb_pixel_total : 1515 time to create 1 rle with old method : 0.002276897430419922 length of segment : 39 time for calcul the mask position with numpy : 5.936622619628906e-05 nb_pixel_total : 340 time to create 1 rle with old method : 0.0005986690521240234 length of segment : 24 time for calcul the mask position with numpy : 7.510185241699219e-05 nb_pixel_total : 1720 time to create 1 rle with old method : 0.0024394989013671875 length of segment : 51 time for calcul the mask position with numpy : 5.364418029785156e-05 nb_pixel_total : 382 time to create 1 rle with old method : 0.000576019287109375 length of segment : 31 time for calcul the mask position with numpy : 4.3392181396484375e-05 nb_pixel_total : 543 time to create 1 rle with old method : 0.0007050037384033203 length of segment : 29 time for calcul the mask position with numpy : 4.220008850097656e-05 nb_pixel_total : 327 time to create 1 rle with old method : 0.00045418739318847656 length of segment : 20 time for calcul the mask position with numpy : 8.463859558105469e-05 nb_pixel_total : 2793 time to create 1 rle with old method : 0.0034224987030029297 length of segment : 75 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -113.49297 max: 149.57266 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 5.435943603515625e-05 nb_pixel_total : 933 time to create 1 rle with old method : 0.0012555122375488281 length of segment : 53 time for calcul the mask position with numpy : 0.0001246929168701172 nb_pixel_total : 2401 time to create 1 rle with old method : 0.003955364227294922 length of segment : 47 time for calcul the mask position with numpy : 5.459785461425781e-05 nb_pixel_total : 832 time to create 1 rle with old method : 0.0011060237884521484 length of segment : 36 time for calcul the mask position with numpy : 3.600120544433594e-05 nb_pixel_total : 127 time to create 1 rle with old method : 0.00037288665771484375 length of segment : 20 time for calcul the mask position with numpy : 7.605552673339844e-05 nb_pixel_total : 1920 time to create 1 rle with old method : 0.0031752586364746094 length of segment : 53 time for calcul the mask position with numpy : 8.487701416015625e-05 nb_pixel_total : 2271 time to create 1 rle with old method : 0.003591299057006836 length of segment : 50 time for calcul the mask position with numpy : 0.00013518333435058594 nb_pixel_total : 5407 time to create 1 rle with old method : 0.006749868392944336 length of segment : 82 Processing 1 images image shape: (400, 400, 3) min: 1.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.08672 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 10 time for calcul the mask position with numpy : 6.818771362304688e-05 nb_pixel_total : 2466 time to create 1 rle with old method : 0.002932310104370117 length of segment : 71 time for calcul the mask position with numpy : 5.269050598144531e-05 nb_pixel_total : 1192 time to create 1 rle with old method : 0.0014874935150146484 length of segment : 41 time for calcul the mask position with numpy : 5.7697296142578125e-05 nb_pixel_total : 1464 time to create 1 rle with old method : 0.0018126964569091797 length of segment : 57 time for calcul the mask position with numpy : 4.76837158203125e-05 nb_pixel_total : 828 time to create 1 rle with old method : 0.0011730194091796875 length of segment : 39 time for calcul the mask position with numpy : 8.296966552734375e-05 nb_pixel_total : 2038 time to create 1 rle with old method : 0.0024979114532470703 length of segment : 55 time for calcul the mask position with numpy : 0.00011301040649414062 nb_pixel_total : 3249 time to create 1 rle with old method : 0.003894329071044922 length of segment : 60 time for calcul the mask position with numpy : 4.76837158203125e-05 nb_pixel_total : 338 time to create 1 rle with old method : 0.00048351287841796875 length of segment : 19 time for calcul the mask position with numpy : 7.534027099609375e-05 nb_pixel_total : 2026 time to create 1 rle with old method : 0.002603292465209961 length of segment : 53 time for calcul the mask position with numpy : 4.5299530029296875e-05 nb_pixel_total : 114 time to create 1 rle with old method : 0.0002319812774658203 length of segment : 15 time for calcul the mask position with numpy : 7.534027099609375e-05 nb_pixel_total : 2272 time to create 1 rle with old method : 0.0030431747436523438 length of segment : 56 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -116.14141 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 4.38690185546875e-05 nb_pixel_total : 371 time to create 1 rle with old method : 0.000514984130859375 length of segment : 34 time for calcul the mask position with numpy : 4.100799560546875e-05 nb_pixel_total : 181 time to create 1 rle with old method : 0.00030493736267089844 length of segment : 16 time for calcul the mask position with numpy : 6.151199340820312e-05 nb_pixel_total : 1570 time to create 1 rle with old method : 0.0020093917846679688 length of segment : 63 time for calcul the mask position with numpy : 5.14984130859375e-05 nb_pixel_total : 402 time to create 1 rle with old method : 0.0005733966827392578 length of segment : 31 time for calcul the mask position with numpy : 3.981590270996094e-05 nb_pixel_total : 315 time to create 1 rle with old method : 0.0005104541778564453 length of segment : 18 time for calcul the mask position with numpy : 5.888938903808594e-05 nb_pixel_total : 443 time to create 1 rle with old method : 0.0007250308990478516 length of segment : 29 Processing 1 images image shape: (400, 160, 3) min: 7.00000 max: 231.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 106.99844 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0002415180206298828 nb_pixel_total : 13597 time to create 1 rle with old method : 0.016439199447631836 length of segment : 252 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -121.03203 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 14 time for calcul the mask position with numpy : 6.365776062011719e-05 nb_pixel_total : 1065 time to create 1 rle with old method : 0.0013391971588134766 length of segment : 45 time for calcul the mask position with numpy : 4.5299530029296875e-05 nb_pixel_total : 432 time to create 1 rle with old method : 0.0006151199340820312 length of segment : 23 time for calcul the mask position with numpy : 7.081031799316406e-05 nb_pixel_total : 1101 time to create 1 rle with old method : 0.002010345458984375 length of segment : 35 time for calcul the mask position with numpy : 5.4836273193359375e-05 nb_pixel_total : 421 time to create 1 rle with old method : 0.0007803440093994141 length of segment : 31 time for calcul the mask position with numpy : 0.0003273487091064453 nb_pixel_total : 12737 time to create 1 rle with old method : 0.020771503448486328 length of segment : 236 time for calcul the mask position with numpy : 0.00014448165893554688 nb_pixel_total : 3241 time to create 1 rle with old method : 0.0064122676849365234 length of segment : 74 time for calcul the mask position with numpy : 9.751319885253906e-05 nb_pixel_total : 688 time to create 1 rle with old method : 0.0010607242584228516 length of segment : 46 time for calcul the mask position with numpy : 5.316734313964844e-05 nb_pixel_total : 511 time to create 1 rle with old method : 0.0007395744323730469 length of segment : 78 time for calcul the mask position with numpy : 5.91278076171875e-05 nb_pixel_total : 966 time to create 1 rle with old method : 0.0013201236724853516 length of segment : 31 time for calcul the mask position with numpy : 4.7206878662109375e-05 nb_pixel_total : 684 time to create 1 rle with old method : 0.0009243488311767578 length of segment : 36 time for calcul the mask position with numpy : 0.00013399124145507812 nb_pixel_total : 6479 time to create 1 rle with old method : 0.007979869842529297 length of segment : 108 time for calcul the mask position with numpy : 0.0003604888916015625 nb_pixel_total : 19129 time to create 1 rle with old method : 0.022237777709960938 length of segment : 165 time for calcul the mask position with numpy : 9.298324584960938e-05 nb_pixel_total : 796 time to create 1 rle with old method : 0.0011861324310302734 length of segment : 45 time for calcul the mask position with numpy : 6.175041198730469e-05 nb_pixel_total : 1211 time to create 1 rle with old method : 0.0015769004821777344 length of segment : 50 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -122.47344 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 13 time for calcul the mask position with numpy : 7.128715515136719e-05 nb_pixel_total : 445 time to create 1 rle with old method : 0.0006306171417236328 length of segment : 46 time for calcul the mask position with numpy : 5.6743621826171875e-05 nb_pixel_total : 427 time to create 1 rle with old method : 0.0005478858947753906 length of segment : 49 time for calcul the mask position with numpy : 3.4809112548828125e-05 nb_pixel_total : 97 time to create 1 rle with old method : 0.0001914501190185547 length of segment : 18 time for calcul the mask position with numpy : 0.00010776519775390625 nb_pixel_total : 3785 time to create 1 rle with old method : 0.004794120788574219 length of segment : 84 time for calcul the mask position with numpy : 8.845329284667969e-05 nb_pixel_total : 2479 time to create 1 rle with old method : 0.0034143924713134766 length of segment : 63 time for calcul the mask position with numpy : 0.0006825923919677734 nb_pixel_total : 47559 time to create 1 rle with old method : 0.05610299110412598 length of segment : 673 time for calcul the mask position with numpy : 7.081031799316406e-05 nb_pixel_total : 359 time to create 1 rle with old method : 0.0005333423614501953 length of segment : 25 time for calcul the mask position with numpy : 3.933906555175781e-05 nb_pixel_total : 299 time to create 1 rle with old method : 0.00042438507080078125 length of segment : 25 time for calcul the mask position with numpy : 4.2438507080078125e-05 nb_pixel_total : 397 time to create 1 rle with old method : 0.0005640983581542969 length of segment : 27 time for calcul the mask position with numpy : 5.4836273193359375e-05 nb_pixel_total : 530 time to create 1 rle with old method : 0.0008873939514160156 length of segment : 22 time for calcul the mask position with numpy : 5.245208740234375e-05 nb_pixel_total : 1125 time to create 1 rle with old method : 0.0014579296112060547 length of segment : 32 time for calcul the mask position with numpy : 6.532669067382812e-05 nb_pixel_total : 1144 time to create 1 rle with old method : 0.001528024673461914 length of segment : 34 time for calcul the mask position with numpy : 4.696846008300781e-05 nb_pixel_total : 439 time to create 1 rle with old method : 0.0006341934204101562 length of segment : 21 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -108.66484 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 0.00012874603271484375 nb_pixel_total : 1164 time to create 1 rle with old method : 0.0020394325256347656 length of segment : 76 time for calcul the mask position with numpy : 9.72747802734375e-05 nb_pixel_total : 1911 time to create 1 rle with old method : 0.0033092498779296875 length of segment : 45 time for calcul the mask position with numpy : 0.00017380714416503906 nb_pixel_total : 5294 time to create 1 rle with old method : 0.008997678756713867 length of segment : 119 time for calcul the mask position with numpy : 0.00015616416931152344 nb_pixel_total : 3565 time to create 1 rle with old method : 0.0065441131591796875 length of segment : 70 time for calcul the mask position with numpy : 6.628036499023438e-05 nb_pixel_total : 364 time to create 1 rle with old method : 0.0007379055023193359 length of segment : 29 time for calcul the mask position with numpy : 5.173683166503906e-05 nb_pixel_total : 270 time to create 1 rle with old method : 0.0005242824554443359 length of segment : 22 time for calcul the mask position with numpy : 4.172325134277344e-05 nb_pixel_total : 193 time to create 1 rle with old method : 0.0003657341003417969 length of segment : 21 time for calcul the mask position with numpy : 0.00043845176696777344 nb_pixel_total : 5157 time to create 1 rle with old method : 0.00865483283996582 length of segment : 122 Processing 1 images image shape: (400, 400, 3) min: 27.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -92.77422 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 4.935264587402344e-05 nb_pixel_total : 421 time to create 1 rle with old method : 0.0006031990051269531 length of segment : 23 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 63 time to create 1 rle with old method : 0.00014591217041015625 length of segment : 7 time for calcul the mask position with numpy : 7.486343383789062e-05 nb_pixel_total : 1300 time to create 1 rle with old method : 0.0016293525695800781 length of segment : 99 Processing 1 images image shape: (400, 400, 3) min: 28.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -90.52813 max: 150.88516 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 0.0004601478576660156 nb_pixel_total : 39334 time to create 1 rle with old method : 0.044635772705078125 length of segment : 258 time for calcul the mask position with numpy : 0.00011849403381347656 nb_pixel_total : 3068 time to create 1 rle with old method : 0.0037393569946289062 length of segment : 77 time for calcul the mask position with numpy : 5.984306335449219e-05 nb_pixel_total : 458 time to create 1 rle with old method : 0.0007071495056152344 length of segment : 42 Processing 1 images image shape: (400, 400, 3) min: 2.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -112.93438 max: 145.18594 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 4.315376281738281e-05 nb_pixel_total : 377 time to create 1 rle with old method : 0.0005438327789306641 length of segment : 30 time for calcul the mask position with numpy : 3.695487976074219e-05 nb_pixel_total : 282 time to create 1 rle with old method : 0.00040149688720703125 length of segment : 21 time for calcul the mask position with numpy : 5.412101745605469e-05 nb_pixel_total : 478 time to create 1 rle with old method : 0.0007116794586181641 length of segment : 66 time for calcul the mask position with numpy : 6.794929504394531e-05 nb_pixel_total : 134 time to create 1 rle with old method : 0.00021505355834960938 length of segment : 20 time for calcul the mask position with numpy : 5.14984130859375e-05 nb_pixel_total : 210 time to create 1 rle with old method : 0.0003566741943359375 length of segment : 20 time for calcul the mask position with numpy : 4.6253204345703125e-05 nb_pixel_total : 298 time to create 1 rle with old method : 0.0005652904510498047 length of segment : 35 Processing 1 images image shape: (400, 160, 3) min: 3.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.10391 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (240, 400, 3) min: 34.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 292 time to create 1 rle with old method : 0.0004191398620605469 length of segment : 23 time for calcul the mask position with numpy : 4.553794860839844e-05 nb_pixel_total : 828 time to create 1 rle with old method : 0.0010540485382080078 length of segment : 93 time for calcul the mask position with numpy : 3.647804260253906e-05 nb_pixel_total : 387 time to create 1 rle with old method : 0.0005514621734619141 length of segment : 21 time for calcul the mask position with numpy : 3.24249267578125e-05 nb_pixel_total : 114 time to create 1 rle with old method : 0.0001709461212158203 length of segment : 29 time for calcul the mask position with numpy : 3.814697265625e-05 nb_pixel_total : 221 time to create 1 rle with old method : 0.00028824806213378906 length of segment : 57 time for calcul the mask position with numpy : 0.0009489059448242188 nb_pixel_total : 80694 time to create 1 rle with old method : 0.09737062454223633 length of segment : 230 Processing 1 images image shape: (240, 400, 3) min: 8.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 9.393692016601562e-05 nb_pixel_total : 1033 time to create 1 rle with old method : 0.002503633499145508 length of segment : 68 time for calcul the mask position with numpy : 8.344650268554688e-05 nb_pixel_total : 224 time to create 1 rle with old method : 0.00046753883361816406 length of segment : 53 time for calcul the mask position with numpy : 5.793571472167969e-05 nb_pixel_total : 572 time to create 1 rle with old method : 0.0010712146759033203 length of segment : 37 time for calcul the mask position with numpy : 0.00017690658569335938 nb_pixel_total : 4319 time to create 1 rle with old method : 0.007598400115966797 length of segment : 88 time for calcul the mask position with numpy : 0.00011444091796875 nb_pixel_total : 1866 time to create 1 rle with old method : 0.003654956817626953 length of segment : 27 time for calcul the mask position with numpy : 8.797645568847656e-05 nb_pixel_total : 439 time to create 1 rle with old method : 0.0010480880737304688 length of segment : 34 time for calcul the mask position with numpy : 0.00010633468627929688 nb_pixel_total : 1894 time to create 1 rle with old method : 0.002509593963623047 length of segment : 63 time for calcul the mask position with numpy : 7.343292236328125e-05 nb_pixel_total : 1883 time to create 1 rle with old method : 0.0024046897888183594 length of segment : 47 Processing 1 images image shape: (240, 400, 3) min: 8.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.82656 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 3.814697265625e-05 nb_pixel_total : 64 time to create 1 rle with old method : 0.00018215179443359375 length of segment : 16 time for calcul the mask position with numpy : 0.00029158592224121094 nb_pixel_total : 19811 time to create 1 rle with old method : 0.027254104614257812 length of segment : 151 time for calcul the mask position with numpy : 9.322166442871094e-05 nb_pixel_total : 3531 time to create 1 rle with old method : 0.004464149475097656 length of segment : 55 Processing 1 images image shape: (240, 400, 3) min: 24.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.39687 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 4.744529724121094e-05 nb_pixel_total : 918 time to create 1 rle with old method : 0.0011866092681884766 length of segment : 41 Processing 1 images image shape: (240, 400, 3) min: 22.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 148.91641 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 6.222724914550781e-05 nb_pixel_total : 73 time to create 1 rle with old method : 0.0002205371856689453 length of segment : 21 time for calcul the mask position with numpy : 5.7220458984375e-05 nb_pixel_total : 259 time to create 1 rle with old method : 0.0004143714904785156 length of segment : 17 time for calcul the mask position with numpy : 5.650520324707031e-05 nb_pixel_total : 789 time to create 1 rle with old method : 0.001123666763305664 length of segment : 39 time for calcul the mask position with numpy : 0.0004754066467285156 nb_pixel_total : 40773 time to create 1 rle with old method : 0.049813270568847656 length of segment : 130 Processing 1 images image shape: (240, 400, 3) min: 21.00000 max: 224.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 116.26797 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0004055500030517578 nb_pixel_total : 32459 time to create 1 rle with old method : 0.038278818130493164 length of segment : 114 Processing 1 images image shape: (240, 160, 3) min: 46.00000 max: 188.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 81.25735 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 3.8623809814453125e-05 nb_pixel_total : 93 time to create 1 rle with old method : 0.0001690387725830078 length of segment : 15 Detection mask done ! Trying to reset tf kernel 3622612 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 888 tf kernel not reseted sub process len(results) : 174 len(list_Values) 174 None max_time_sub_proc : 3600 parent process len(results) : 0 len(list_Values) 174 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 181 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Catched exception ! Connect or reconnect ! Number saved : None batch 1 Loaded 181 chid ids of type : 4228 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 9769 save missing photos in datou_result : time spend for datou_step_exec : 23.336345434188843 time spend to save output : 0.9631919860839844 total time spend for step 1 : 24.299537420272827 step2:brightness Wed Mar 5 06:20:53 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness toutes les photos sont déjà traitées, on saute les calculs Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 1 time used for this insertion : 0.008068561553955078 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 1 time used for this insertion : 0.007944107055664062 save missing photos in datou_result : time spend for datou_step_exec : 0.031969308853149414 time spend to save output : 0.02003002166748047 total time spend for step 2 : 0.05199933052062988 step3:blur_detection Wed Mar 5 06:20:53 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection toutes les photos sont déjà traitées, on saute les calculs Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 1 time used for this insertion : 0.008004426956176758 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 1 time used for this insertion : 0.007943868637084961 save missing photos in datou_result : time spend for datou_step_exec : 0.021577119827270508 time spend to save output : 0.02029109001159668 total time spend for step 3 : 0.04186820983886719 step4:rle_unique_nms_with_priority Wed Mar 5 06:20:53 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 181 chid ids of type : 4228 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 181 nb_hashtags : 6 time to prepare the origin masks : 3.889739990234375 time for calcul the mask position with numpy : 0.3886749744415283 nb_pixel_total : 3203410 time to create 1 rle with new method : 0.9331109523773193 time for calcul the mask position with numpy : 0.015521764755249023 nb_pixel_total : 151 time to create 1 rle with old method : 0.00022101402282714844 time for calcul the mask position with numpy : 0.015331506729125977 nb_pixel_total : 108 time to create 1 rle with old method : 0.0001678466796875 time for calcul the mask position with numpy : 0.015243768692016602 nb_pixel_total : 429 time to create 1 rle with old method : 0.0005269050598144531 time for calcul the mask position with numpy : 0.015292882919311523 nb_pixel_total : 546 time to create 1 rle with old method : 0.0006532669067382812 time for calcul the mask position with numpy : 0.015253782272338867 nb_pixel_total : 75 time to create 1 rle with old method : 0.00011014938354492188 time for calcul the mask position with numpy : 0.015333414077758789 nb_pixel_total : 378 time to create 1 rle with old method : 0.00046896934509277344 time for calcul the mask position with numpy : 0.01533055305480957 nb_pixel_total : 328 time to create 1 rle with old method : 0.0004277229309082031 time for calcul the mask position with numpy : 0.015458822250366211 nb_pixel_total : 402 time to create 1 rle with old method : 0.00047850608825683594 time for calcul the mask position with numpy : 0.01503896713256836 nb_pixel_total : 306 time to create 1 rle with old method : 0.0003829002380371094 time for calcul the mask position with numpy : 0.014935016632080078 nb_pixel_total : 464 time to create 1 rle with old method : 0.0005362033843994141 time for calcul the mask position with numpy : 0.01538991928100586 nb_pixel_total : 203 time to create 1 rle with old method : 0.00028634071350097656 time for calcul the mask position with numpy : 0.0159146785736084 nb_pixel_total : 14050 time to create 1 rle with old method : 0.01604437828063965 time for calcul the mask position with numpy : 0.015468835830688477 nb_pixel_total : 513 time to create 1 rle with old method : 0.0006082057952880859 time for calcul the mask position with numpy : 0.015073060989379883 nb_pixel_total : 761 time to create 1 rle with old method : 0.0008711814880371094 time for calcul the mask position with numpy : 0.015141487121582031 nb_pixel_total : 412 time to create 1 rle with old method : 0.0005171298980712891 time for calcul the mask position with numpy : 0.01733708381652832 nb_pixel_total : 334 time to create 1 rle with old method : 0.0004074573516845703 time for calcul the mask position with numpy : 0.015076875686645508 nb_pixel_total : 2024 time to create 1 rle with old method : 0.0022966861724853516 time for calcul the mask position with numpy : 0.014960050582885742 nb_pixel_total : 13 time to create 1 rle with old method : 4.57763671875e-05 time for calcul the mask position with numpy : 0.014908313751220703 nb_pixel_total : 884 time to create 1 rle with old method : 0.0010344982147216797 time for calcul the mask position with numpy : 0.014905452728271484 nb_pixel_total : 13 time to create 1 rle with old method : 2.9802322387695312e-05 time for calcul the mask position with numpy : 0.014865875244140625 nb_pixel_total : 312 time to create 1 rle with old method : 0.0003788471221923828 time for calcul the mask position with numpy : 0.014928340911865234 nb_pixel_total : 843 time to create 1 rle with old method : 0.0009932518005371094 time for calcul the mask position with numpy : 0.014935731887817383 nb_pixel_total : 726 time to create 1 rle with old method : 0.0009305477142333984 time for calcul the mask position with numpy : 0.014924049377441406 nb_pixel_total : 715 time to create 1 rle with old method : 0.0008778572082519531 time for calcul the mask position with numpy : 0.015098810195922852 nb_pixel_total : 252 time to create 1 rle with old method : 0.00033783912658691406 time for calcul the mask position with numpy : 0.015134572982788086 nb_pixel_total : 239 time to create 1 rle with old method : 0.0003390312194824219 time for calcul the mask position with numpy : 0.015024185180664062 nb_pixel_total : 369 time to create 1 rle with old method : 0.0004687309265136719 time for calcul the mask position with numpy : 0.015251398086547852 nb_pixel_total : 504 time to create 1 rle with old method : 0.00063323974609375 time for calcul the mask position with numpy : 0.015203475952148438 nb_pixel_total : 94 time to create 1 rle with old method : 0.00018930435180664062 time for calcul the mask position with numpy : 0.014922142028808594 nb_pixel_total : 1037 time to create 1 rle with old method : 0.0012543201446533203 time for calcul the mask position with numpy : 0.014953136444091797 nb_pixel_total : 79 time to create 1 rle with old method : 0.0001308917999267578 time for calcul the mask position with numpy : 0.01487588882446289 nb_pixel_total : 755 time to create 1 rle with old method : 0.001409769058227539 time for calcul the mask position with numpy : 0.014904499053955078 nb_pixel_total : 187 time to create 1 rle with old method : 0.00023555755615234375 time for calcul the mask position with numpy : 0.01493692398071289 nb_pixel_total : 1092 time to create 1 rle with old method : 0.0013229846954345703 time for calcul the mask position with numpy : 0.014906883239746094 nb_pixel_total : 56 time to create 1 rle with old method : 8.869171142578125e-05 time for calcul the mask position with numpy : 0.015233516693115234 nb_pixel_total : 243 time to create 1 rle with old method : 0.00031495094299316406 time for calcul the mask position with numpy : 0.015227794647216797 nb_pixel_total : 993 time to create 1 rle with old method : 0.0012156963348388672 time for calcul the mask position with numpy : 0.015313386917114258 nb_pixel_total : 6 time to create 1 rle with old method : 6.389617919921875e-05 time for calcul the mask position with numpy : 0.014964818954467773 nb_pixel_total : 161 time to create 1 rle with old method : 0.0002574920654296875 time for calcul the mask position with numpy : 0.014857292175292969 nb_pixel_total : 1656 time to create 1 rle with old method : 0.0019023418426513672 time for calcul the mask position with numpy : 0.01494741439819336 nb_pixel_total : 969 time to create 1 rle with old method : 0.0011365413665771484 time for calcul the mask position with numpy : 0.014909744262695312 nb_pixel_total : 255 time to create 1 rle with old method : 0.0003077983856201172 time for calcul the mask position with numpy : 0.014896392822265625 nb_pixel_total : 2378 time to create 1 rle with old method : 0.0027298927307128906 time for calcul the mask position with numpy : 0.014921426773071289 nb_pixel_total : 1487 time to create 1 rle with old method : 0.0018053054809570312 time for calcul the mask position with numpy : 0.014959335327148438 nb_pixel_total : 434 time to create 1 rle with old method : 0.0005519390106201172 time for calcul the mask position with numpy : 0.015167236328125 nb_pixel_total : 192 time to create 1 rle with old method : 0.0002474784851074219 time for calcul the mask position with numpy : 0.015214204788208008 nb_pixel_total : 418 time to create 1 rle with old method : 0.0005221366882324219 time for calcul the mask position with numpy : 0.015230894088745117 nb_pixel_total : 179 time to create 1 rle with old method : 0.0002684593200683594 time for calcul the mask position with numpy : 0.015269756317138672 nb_pixel_total : 406 time to create 1 rle with old method : 0.0004901885986328125 time for calcul the mask position with numpy : 0.014892101287841797 nb_pixel_total : 939 time to create 1 rle with old method : 0.0011641979217529297 time for calcul the mask position with numpy : 0.015263557434082031 nb_pixel_total : 53 time to create 1 rle with old method : 0.00016880035400390625 time for calcul the mask position with numpy : 0.015217304229736328 nb_pixel_total : 1920 time to create 1 rle with old method : 0.0023038387298583984 time for calcul the mask position with numpy : 0.014881372451782227 nb_pixel_total : 441 time to create 1 rle with old method : 0.00058746337890625 time for calcul the mask position with numpy : 0.01508641242980957 nb_pixel_total : 1515 time to create 1 rle with old method : 0.0018243789672851562 time for calcul the mask position with numpy : 0.01496577262878418 nb_pixel_total : 1184 time to create 1 rle with old method : 0.0013928413391113281 time for calcul the mask position with numpy : 0.014879703521728516 nb_pixel_total : 5407 time to create 1 rle with old method : 0.006296396255493164 time for calcul the mask position with numpy : 0.014914751052856445 nb_pixel_total : 322 time to create 1 rle with old method : 0.00038123130798339844 time for calcul the mask position with numpy : 0.014909029006958008 nb_pixel_total : 501 time to create 1 rle with old method : 0.0006148815155029297 time for calcul the mask position with numpy : 0.014935970306396484 nb_pixel_total : 382 time to create 1 rle with old method : 0.0004715919494628906 time for calcul the mask position with numpy : 0.01492166519165039 nb_pixel_total : 338 time to create 1 rle with old method : 0.0004086494445800781 time for calcul the mask position with numpy : 0.014884233474731445 nb_pixel_total : 2466 time to create 1 rle with old method : 0.002877950668334961 time for calcul the mask position with numpy : 0.014896392822265625 nb_pixel_total : 2798 time to create 1 rle with old method : 0.003307819366455078 time for calcul the mask position with numpy : 0.014997482299804688 nb_pixel_total : 142 time to create 1 rle with old method : 0.00022292137145996094 time for calcul the mask position with numpy : 0.014919281005859375 nb_pixel_total : 268 time to create 1 rle with old method : 0.00033020973205566406 time for calcul the mask position with numpy : 0.015339136123657227 nb_pixel_total : 933 time to create 1 rle with old method : 0.001171112060546875 time for calcul the mask position with numpy : 0.01603865623474121 nb_pixel_total : 614 time to create 1 rle with old method : 0.0009522438049316406 time for calcul the mask position with numpy : 0.016048192977905273 nb_pixel_total : 315 time to create 1 rle with old method : 0.00045228004455566406 time for calcul the mask position with numpy : 0.016109466552734375 nb_pixel_total : 832 time to create 1 rle with old method : 0.0011050701141357422 time for calcul the mask position with numpy : 0.016059398651123047 nb_pixel_total : 2026 time to create 1 rle with old method : 0.002466440200805664 time for calcul the mask position with numpy : 0.016185522079467773 nb_pixel_total : 335 time to create 1 rle with old method : 0.0005648136138916016 time for calcul the mask position with numpy : 0.015356302261352539 nb_pixel_total : 340 time to create 1 rle with old method : 0.0004429817199707031 time for calcul the mask position with numpy : 0.015232324600219727 nb_pixel_total : 1262 time to create 1 rle with old method : 0.0015294551849365234 time for calcul the mask position with numpy : 0.015293121337890625 nb_pixel_total : 451 time to create 1 rle with old method : 0.0005486011505126953 time for calcul the mask position with numpy : 0.015200376510620117 nb_pixel_total : 327 time to create 1 rle with old method : 0.0003991127014160156 time for calcul the mask position with numpy : 0.015213727951049805 nb_pixel_total : 1720 time to create 1 rle with old method : 0.0020112991333007812 time for calcul the mask position with numpy : 0.015193939208984375 nb_pixel_total : 8076 time to create 1 rle with old method : 0.009508609771728516 time for calcul the mask position with numpy : 0.015197277069091797 nb_pixel_total : 314 time to create 1 rle with old method : 0.0005462169647216797 time for calcul the mask position with numpy : 0.015406370162963867 nb_pixel_total : 543 time to create 1 rle with old method : 0.0006608963012695312 time for calcul the mask position with numpy : 0.015271425247192383 nb_pixel_total : 402 time to create 1 rle with old method : 0.00048732757568359375 time for calcul the mask position with numpy : 0.015256643295288086 nb_pixel_total : 2038 time to create 1 rle with old method : 0.002353668212890625 time for calcul the mask position with numpy : 0.015250921249389648 nb_pixel_total : 2401 time to create 1 rle with old method : 0.0027408599853515625 time for calcul the mask position with numpy : 0.015381574630737305 nb_pixel_total : 13597 time to create 1 rle with old method : 0.015333175659179688 time for calcul the mask position with numpy : 0.015017032623291016 nb_pixel_total : 181 time to create 1 rle with old method : 0.0002446174621582031 time for calcul the mask position with numpy : 0.01499795913696289 nb_pixel_total : 1879 time to create 1 rle with old method : 0.0022182464599609375 time for calcul the mask position with numpy : 0.0150299072265625 nb_pixel_total : 15059 time to create 1 rle with old method : 0.017211437225341797 time for calcul the mask position with numpy : 0.015038490295410156 nb_pixel_total : 1474 time to create 1 rle with old method : 0.0017044544219970703 time for calcul the mask position with numpy : 0.014907121658325195 nb_pixel_total : 1464 time to create 1 rle with old method : 0.0017132759094238281 time for calcul the mask position with numpy : 0.015001773834228516 nb_pixel_total : 424 time to create 1 rle with old method : 0.0006310939788818359 time for calcul the mask position with numpy : 0.014941930770874023 nb_pixel_total : 5178 time to create 1 rle with old method : 0.005974292755126953 time for calcul the mask position with numpy : 0.014963865280151367 nb_pixel_total : 74 time to create 1 rle with old method : 0.00021696090698242188 time for calcul the mask position with numpy : 0.014986515045166016 nb_pixel_total : 3249 time to create 1 rle with old method : 0.003758668899536133 time for calcul the mask position with numpy : 0.015035629272460938 nb_pixel_total : 431 time to create 1 rle with old method : 0.00054168701171875 time for calcul the mask position with numpy : 0.01522374153137207 nb_pixel_total : 241 time to create 1 rle with old method : 0.0003643035888671875 time for calcul the mask position with numpy : 0.015143394470214844 nb_pixel_total : 3452 time to create 1 rle with old method : 0.004049777984619141 time for calcul the mask position with numpy : 0.015138387680053711 nb_pixel_total : 2432 time to create 1 rle with old method : 0.002832174301147461 time for calcul the mask position with numpy : 0.015238523483276367 nb_pixel_total : 2793 time to create 1 rle with old method : 0.0032868385314941406 time for calcul the mask position with numpy : 0.015985488891601562 nb_pixel_total : 1570 time to create 1 rle with old method : 0.0018668174743652344 time for calcul the mask position with numpy : 0.015349388122558594 nb_pixel_total : 1036 time to create 1 rle with old method : 0.001348257064819336 time for calcul the mask position with numpy : 0.015286445617675781 nb_pixel_total : 738 time to create 1 rle with old method : 0.0008924007415771484 time for calcul the mask position with numpy : 0.015346527099609375 nb_pixel_total : 48 time to create 1 rle with old method : 0.00011849403381347656 time for calcul the mask position with numpy : 0.015352964401245117 nb_pixel_total : 1825 time to create 1 rle with old method : 0.002131223678588867 time for calcul the mask position with numpy : 0.015213966369628906 nb_pixel_total : 2287 time to create 1 rle with old method : 0.0026514530181884766 time for calcul the mask position with numpy : 0.015226602554321289 nb_pixel_total : 114 time to create 1 rle with old method : 0.00017571449279785156 time for calcul the mask position with numpy : 0.015424489974975586 nb_pixel_total : 371 time to create 1 rle with old method : 0.0004634857177734375 time for calcul the mask position with numpy : 0.015398979187011719 nb_pixel_total : 828 time to create 1 rle with old method : 0.0010077953338623047 time for calcul the mask position with numpy : 0.015288591384887695 nb_pixel_total : 127 time to create 1 rle with old method : 0.0002181529998779297 time for calcul the mask position with numpy : 0.015368461608886719 nb_pixel_total : 97 time to create 1 rle with old method : 0.0001659393310546875 time for calcul the mask position with numpy : 0.015269756317138672 nb_pixel_total : 298 time to create 1 rle with old method : 0.0004050731658935547 time for calcul the mask position with numpy : 0.018007278442382812 nb_pixel_total : 63 time to create 1 rle with old method : 0.00020694732666015625 time for calcul the mask position with numpy : 0.020398616790771484 nb_pixel_total : 6454 time to create 1 rle with old method : 0.01169133186340332 time for calcul the mask position with numpy : 0.02181386947631836 nb_pixel_total : 39334 time to create 1 rle with old method : 0.07069826126098633 time for calcul the mask position with numpy : 0.017261028289794922 nb_pixel_total : 669 time to create 1 rle with old method : 0.0008823871612548828 time for calcul the mask position with numpy : 0.015465021133422852 nb_pixel_total : 182 time to create 1 rle with old method : 0.00031948089599609375 time for calcul the mask position with numpy : 0.015547037124633789 nb_pixel_total : 12736 time to create 1 rle with old method : 0.014691829681396484 time for calcul the mask position with numpy : 0.016578197479248047 nb_pixel_total : 149 time to create 1 rle with old method : 0.0003535747528076172 time for calcul the mask position with numpy : 0.01818370819091797 nb_pixel_total : 47 time to create 1 rle with old method : 0.00017499923706054688 time for calcul the mask position with numpy : 0.015608549118041992 nb_pixel_total : 5294 time to create 1 rle with old method : 0.006418466567993164 time for calcul the mask position with numpy : 0.015056610107421875 nb_pixel_total : 193 time to create 1 rle with old method : 0.0002551078796386719 time for calcul the mask position with numpy : 0.014974832534790039 nb_pixel_total : 359 time to create 1 rle with old method : 0.00045943260192871094 time for calcul the mask position with numpy : 0.014913558959960938 nb_pixel_total : 445 time to create 1 rle with old method : 0.0005495548248291016 time for calcul the mask position with numpy : 0.015071868896484375 nb_pixel_total : 432 time to create 1 rle with old method : 0.0005412101745605469 time for calcul the mask position with numpy : 0.015002965927124023 nb_pixel_total : 210 time to create 1 rle with old method : 0.00027942657470703125 time for calcul the mask position with numpy : 0.015004158020019531 nb_pixel_total : 377 time to create 1 rle with old method : 0.0004744529724121094 time for calcul the mask position with numpy : 0.015230178833007812 nb_pixel_total : 382 time to create 1 rle with old method : 0.0005359649658203125 time for calcul the mask position with numpy : 0.015179634094238281 nb_pixel_total : 2479 time to create 1 rle with old method : 0.002922534942626953 time for calcul the mask position with numpy : 0.01510763168334961 nb_pixel_total : 1300 time to create 1 rle with old method : 0.0015354156494140625 time for calcul the mask position with numpy : 0.015382766723632812 nb_pixel_total : 299 time to create 1 rle with old method : 0.0003962516784667969 time for calcul the mask position with numpy : 0.015231847763061523 nb_pixel_total : 270 time to create 1 rle with old method : 0.00036334991455078125 time for calcul the mask position with numpy : 0.015154123306274414 nb_pixel_total : 478 time to create 1 rle with old method : 0.0006122589111328125 time for calcul the mask position with numpy : 0.01571488380432129 nb_pixel_total : 47201 time to create 1 rle with old method : 0.05548572540283203 time for calcul the mask position with numpy : 0.015385150909423828 nb_pixel_total : 74 time to create 1 rle with old method : 0.00013256072998046875 time for calcul the mask position with numpy : 0.01516580581665039 nb_pixel_total : 421 time to create 1 rle with old method : 0.0005059242248535156 time for calcul the mask position with numpy : 0.01511836051940918 nb_pixel_total : 1120 time to create 1 rle with old method : 0.00130462646484375 time for calcul the mask position with numpy : 0.014998435974121094 nb_pixel_total : 688 time to create 1 rle with old method : 0.0008153915405273438 time for calcul the mask position with numpy : 0.015070676803588867 nb_pixel_total : 1911 time to create 1 rle with old method : 0.002248048782348633 time for calcul the mask position with numpy : 0.014990091323852539 nb_pixel_total : 458 time to create 1 rle with old method : 0.0005753040313720703 time for calcul the mask position with numpy : 0.01502084732055664 nb_pixel_total : 134 time to create 1 rle with old method : 0.00016951560974121094 time for calcul the mask position with numpy : 0.015195131301879883 nb_pixel_total : 966 time to create 1 rle with old method : 0.0011339187622070312 time for calcul the mask position with numpy : 0.014948844909667969 nb_pixel_total : 1164 time to create 1 rle with old method : 0.0013535022735595703 time for calcul the mask position with numpy : 0.014968633651733398 nb_pixel_total : 3241 time to create 1 rle with old method : 0.003737926483154297 time for calcul the mask position with numpy : 0.015172958374023438 nb_pixel_total : 681 time to create 1 rle with old method : 0.0008044242858886719 time for calcul the mask position with numpy : 0.01498723030090332 nb_pixel_total : 6479 time to create 1 rle with old method : 0.00726318359375 time for calcul the mask position with numpy : 0.014982461929321289 nb_pixel_total : 3068 time to create 1 rle with old method : 0.003468036651611328 time for calcul the mask position with numpy : 0.014956235885620117 nb_pixel_total : 427 time to create 1 rle with old method : 0.0005097389221191406 time for calcul the mask position with numpy : 0.015037775039672852 nb_pixel_total : 1101 time to create 1 rle with old method : 0.0012805461883544922 time for calcul the mask position with numpy : 0.014918804168701172 nb_pixel_total : 419 time to create 1 rle with old method : 0.0005116462707519531 time for calcul the mask position with numpy : 0.014961481094360352 nb_pixel_total : 364 time to create 1 rle with old method : 0.00045871734619140625 time for calcul the mask position with numpy : 0.015074491500854492 nb_pixel_total : 3785 time to create 1 rle with old method : 0.004395723342895508 time for calcul the mask position with numpy : 0.015011787414550781 nb_pixel_total : 3565 time to create 1 rle with old method : 0.004099607467651367 time for calcul the mask position with numpy : 0.015150785446166992 nb_pixel_total : 282 time to create 1 rle with old method : 0.00038361549377441406 time for calcul the mask position with numpy : 0.015354633331298828 nb_pixel_total : 1211 time to create 1 rle with old method : 0.0020427703857421875 time for calcul the mask position with numpy : 0.017256736755371094 nb_pixel_total : 1065 time to create 1 rle with old method : 0.001787424087524414 time for calcul the mask position with numpy : 0.01726984977722168 nb_pixel_total : 530 time to create 1 rle with old method : 0.0010595321655273438 time for calcul the mask position with numpy : 0.017383813858032227 nb_pixel_total : 439 time to create 1 rle with old method : 0.0007641315460205078 time for calcul the mask position with numpy : 0.018413543701171875 nb_pixel_total : 439 time to create 1 rle with old method : 0.0005576610565185547 time for calcul the mask position with numpy : 0.015775203704833984 nb_pixel_total : 158 time to create 1 rle with old method : 0.0002808570861816406 time for calcul the mask position with numpy : 0.016217947006225586 nb_pixel_total : 1611 time to create 1 rle with old method : 0.0026264190673828125 time for calcul the mask position with numpy : 0.01570582389831543 nb_pixel_total : 1866 time to create 1 rle with old method : 0.002329111099243164 time for calcul the mask position with numpy : 0.015417098999023438 nb_pixel_total : 1873 time to create 1 rle with old method : 0.002332448959350586 time for calcul the mask position with numpy : 0.015527725219726562 nb_pixel_total : 4195 time to create 1 rle with old method : 0.004729032516479492 time for calcul the mask position with numpy : 0.01511240005493164 nb_pixel_total : 387 time to create 1 rle with old method : 0.0004899501800537109 time for calcul the mask position with numpy : 0.015378713607788086 nb_pixel_total : 918 time to create 1 rle with old method : 0.0017049312591552734 time for calcul the mask position with numpy : 0.015375375747680664 nb_pixel_total : 19747 time to create 1 rle with old method : 0.022452592849731445 time for calcul the mask position with numpy : 0.01541447639465332 nb_pixel_total : 32459 time to create 1 rle with old method : 0.03641986846923828 time for calcul the mask position with numpy : 0.015385150909423828 nb_pixel_total : 49 time to create 1 rle with old method : 0.0003635883331298828 time for calcul the mask position with numpy : 0.01602935791015625 nb_pixel_total : 79037 time to create 1 rle with old method : 0.08721184730529785 time for calcul the mask position with numpy : 0.015113592147827148 nb_pixel_total : 83 time to create 1 rle with old method : 0.0004048347473144531 time for calcul the mask position with numpy : 0.015063285827636719 nb_pixel_total : 40487 time to create 1 rle with old method : 0.04503178596496582 time for calcul the mask position with numpy : 0.016555309295654297 nb_pixel_total : 215 time to create 1 rle with old method : 0.0003960132598876953 time for calcul the mask position with numpy : 0.017449378967285156 nb_pixel_total : 90 time to create 1 rle with old method : 0.0002701282501220703 time for calcul the mask position with numpy : 0.01751542091369629 nb_pixel_total : 14 time to create 1 rle with old method : 7.653236389160156e-05 time for calcul the mask position with numpy : 0.017154455184936523 nb_pixel_total : 828 time to create 1 rle with old method : 0.0014300346374511719 time for calcul the mask position with numpy : 0.017177104949951172 nb_pixel_total : 259 time to create 1 rle with old method : 0.0004596710205078125 time for calcul the mask position with numpy : 0.01726222038269043 nb_pixel_total : 64 time to create 1 rle with old method : 0.00018930435180664062 time for calcul the mask position with numpy : 0.017354488372802734 nb_pixel_total : 110 time to create 1 rle with old method : 0.00023508071899414062 time for calcul the mask position with numpy : 0.01736283302307129 nb_pixel_total : 1033 time to create 1 rle with old method : 0.0017788410186767578 time for calcul the mask position with numpy : 0.017330408096313477 nb_pixel_total : 292 time to create 1 rle with old method : 0.0005109310150146484 time for calcul the mask position with numpy : 0.017833471298217773 nb_pixel_total : 73 time to create 1 rle with old method : 0.00015282630920410156 time for calcul the mask position with numpy : 0.015435457229614258 nb_pixel_total : 572 time to create 1 rle with old method : 0.000690460205078125 time for calcul the mask position with numpy : 0.015331268310546875 nb_pixel_total : 93 time to create 1 rle with old method : 0.00013756752014160156 time for calcul the mask position with numpy : 0.015259742736816406 nb_pixel_total : 789 time to create 1 rle with old method : 0.0010204315185546875 create new chi : 4.745086431503296 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.016854524612426758 batch 1 Loaded 189 chid ids of type : 4230 Number RLEs to save : 19498 TO DO : save crop sub photo not yet done ! save time : 1.3912007808685303 map_output_result : {1340722929: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1340722929] Looping around the photos to save general results len do output : 1 /1340722929.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2634220') ('4234', '21027618', '1340722929', None, None, None, None, None, '2634220') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.01298975944519043 save_final save missing photos in datou_result : time spend for datou_step_exec : 10.466784000396729 time spend to save output : 0.013294219970703125 total time spend for step 4 : 10.480078220367432 step5:crop_condition Wed Mar 5 06:21:04 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4230 Loading chi in step crop for list_pids : 1 ! batch 1 Loaded 189 chid ids of type : 4230 begin to crop the class : papier param for this class : {'min_score': 0.6} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 111 About to insert : list_path_to_insert length 111 new photo from crops ! About to upload 111 photos upload in portfolio : 4869462 init cache_photo without model_param we have 111 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741152068_3622369 we have uploaded 111 photos in the portfolio 4869462 time of upload the photos Elapsed time : 29.15851616859436 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.6} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 19 About to insert : list_path_to_insert length 19 new photo from crops ! About to upload 19 photos upload in portfolio : 4869462 init cache_photo without model_param we have 19 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741152097_3622369 we have uploaded 19 photos in the portfolio 4869462 time of upload the photos Elapsed time : 5.068280935287476 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.6} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.6} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! About to upload 9 photos upload in portfolio : 4869462 init cache_photo without model_param we have 9 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741152103_3622369 we have uploaded 9 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.7960236072540283 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.6} filtre for class : autre hashtag_id of this class : 494826614 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741152106_3622369 we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.1475234031677246 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.6} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.6} filtre for class : pet_fonce hashtag_id of this class : 2107755900 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741152108_3622369 we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.6987893581390381 we have finished the crop for the class : pet_fonce delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1340722929] Looping around the photos to save general results len do output : 145 /1340947623Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947624Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947625Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947626Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947627Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947628Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947629Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947630Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947631Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947632Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947633Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947634Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947635Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947636Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947638Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947639Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947640Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947641Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947642Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947643Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947644Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947645Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947646Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947647Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947648Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947649Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947650Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947651Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947652Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947653Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947654Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947655Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947656Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947657Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947658Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947659Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947660Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947661Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947662Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947663Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947664Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947665Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947666Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947667Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947668Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947669Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947671Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947672Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947673Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947674Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947675Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947676Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947677Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947678Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947679Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947680Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947681Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947682Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947683Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947684Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947685Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947686Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947687Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947688Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947689Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947690Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947691Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947692Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947693Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947694Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947695Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947696Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947697Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947698Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947699Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947700Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947701Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947702Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947703Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947704Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947705Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947706Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947707Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947708Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947709Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947710Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947711Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947713Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947714Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947715Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947716Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947717Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947718Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947719Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947720Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947721Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947722Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947723Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947724Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947725Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947726Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947727Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947728Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947729Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947730Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947732Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947733Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947734Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947735Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947736Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947737Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947742Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947743Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947744Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947745Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947746Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947747Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947748Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947749Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947750Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947751Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947752Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947753Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947755Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947756Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947757Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947758Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947759Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947760Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947761Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947764Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947765Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947766Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947767Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947768Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947769Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947770Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947771Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947772Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947776Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947777Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947778Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947779Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947780Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340947782Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2634220') ('4234', '21027618', '1340722929', None, None, None, None, None, '2634220') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 436 time used for this insertion : 0.05449938774108887 save_final save missing photos in datou_result : time spend for datou_step_exec : 44.92866134643555 time spend to save output : 0.06000256538391113 total time spend for step 5 : 44.98866391181946 step6:thcl Wed Mar 5 06:21:49 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! we are using the classfication for only one thcl 3237 time to import caffe and check if the image exist : 0.007604122161865234 time to convert the images to numpy array : 0.032089948654174805 time to import caffe and check if the image exist : 0.007300853729248047 time to convert the images to numpy array : 0.03209686279296875 time to import caffe and check if the image exist : 0.006034135818481445 time to convert the images to numpy array : 0.036446332931518555 time to import caffe and check if the image exist : 0.007889270782470703 time to convert the images to numpy array : 0.03634476661682129 time to import caffe and check if the image exist : 0.017771005630493164 time to convert the images to numpy array : 0.03145718574523926 time to import caffe and check if the image exist : 0.015989065170288086 time to convert the images to numpy array : 0.034288644790649414 time to import caffe and check if the image exist : 0.014302730560302734 time to convert the images to numpy array : 0.03716135025024414 time to import caffe and check if the image exist : 0.014124155044555664 time to convert the images to numpy array : 0.037575483322143555 time to import caffe and check if the image exist : 0.016629457473754883 time to convert the images to numpy array : 0.0378117561340332 time to import caffe and check if the image exist : 0.013500690460205078 time to convert the images to numpy array : 0.04275226593017578 total time to convert the images to numpy array : 0.28924059867858887 list photo_ids error: [] list photo_ids correct : [1340947623, 1340947624, 1340947625, 1340947626, 1340947627, 1340947628, 1340947629, 1340947630, 1340947631, 1340947632, 1340947633, 1340947634, 1340947635, 1340947636, 1340947638, 1340947751, 1340947752, 1340947753, 1340947755, 1340947756, 1340947757, 1340947758, 1340947759, 1340947760, 1340947761, 1340947764, 1340947765, 1340947766, 1340947767, 1340947768, 1340947769, 1340947770, 1340947771, 1340947772, 1340947776, 1340947777, 1340947778, 1340947779, 1340947780, 1340947782, 1340947685, 1340947686, 1340947687, 1340947688, 1340947689, 1340947690, 1340947691, 1340947692, 1340947693, 1340947694, 1340947695, 1340947696, 1340947697, 1340947698, 1340947699, 1340947700, 1340947701, 1340947702, 1340947703, 1340947704, 1340947705, 1340947706, 1340947707, 1340947708, 1340947709, 1340947710, 1340947711, 1340947713, 1340947714, 1340947715, 1340947654, 1340947655, 1340947656, 1340947657, 1340947658, 1340947659, 1340947660, 1340947661, 1340947662, 1340947663, 1340947664, 1340947665, 1340947666, 1340947667, 1340947668, 1340947669, 1340947671, 1340947672, 1340947673, 1340947674, 1340947675, 1340947676, 1340947677, 1340947678, 1340947679, 1340947680, 1340947681, 1340947682, 1340947683, 1340947684, 1340947732, 1340947733, 1340947734, 1340947735, 1340947736, 1340947737, 1340947742, 1340947743, 1340947744, 1340947745, 1340947746, 1340947747, 1340947748, 1340947749, 1340947750, 1340947716, 1340947717, 1340947718, 1340947719, 1340947720, 1340947721, 1340947722, 1340947723, 1340947724, 1340947725, 1340947726, 1340947727, 1340947728, 1340947729, 1340947730, 1340947639, 1340947640, 1340947641, 1340947642, 1340947643, 1340947644, 1340947645, 1340947646, 1340947647, 1340947648, 1340947649, 1340947650, 1340947651, 1340947652, 1340947653] number of photos to traite : 145 try to delete the photos incorrect in DB tagging for thcl : 3237 To do loadFromThcl(), then load ParamDescType : thcl3237 thcls : [{'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'}] thcl {'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'} Update svm_hashtag_type_desc : 5561 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) To loadFromThcl() : net_5561 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3185 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_rubbia_refus_2500 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_rubbia_refus_2500 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_rubbia_refus_2500 /data/models_weight/learn_rubbia_refus_2500/caffemodel size_local : 94358479 size in s3 : 94358479 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:16 caffemodel already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt size_local : 32544 size in s3 : 32544 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:15 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:09:52 mean.npy already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/synset_words.txt size_local : 334 size in s3 : 334 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:10:06 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt caffemodel_filename : /data/models_weight/learn_rubbia_refus_2500/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 2966 max_wait_temp : 1 max_wait : 0 dict_keys(['prob', 'pool5']) time used to do the prepocess of the images : 0.6827023029327393 time used to do the prediction : 1.3097548484802246 save descriptor for thcl : 3237 time to traite the descriptors : 0.862541913986206 Catched exception ! Connect or reconnect ! storage_type for insertDescriptorsMulti : 3 To insert : 1340947623 To insert : 1340947624 To insert : 1340947625 To insert : 1340947626 To insert : 1340947627 To insert : 1340947628 To insert : 1340947629 To insert : 1340947630 To insert : 1340947631 To insert : 1340947632 To insert : 1340947633 To insert : 1340947634 To insert : 1340947635 To insert : 1340947636 To insert : 1340947638 To insert : 1340947751 To insert : 1340947752 To insert : 1340947753 To insert : 1340947755 To insert : 1340947756 To insert : 1340947757 To insert : 1340947758 To insert : 1340947759 To insert : 1340947760 To insert : 1340947761 To insert : 1340947764 To insert : 1340947765 To insert : 1340947766 To insert : 1340947767 To insert : 1340947768 To insert : 1340947769 To insert : 1340947770 To insert : 1340947771 To insert : 1340947772 To insert : 1340947776 To insert : 1340947777 To insert : 1340947778 To insert : 1340947779 To insert : 1340947780 To insert : 1340947782 To insert : 1340947685 To insert : 1340947686 To insert : 1340947687 To insert : 1340947688 To insert : 1340947689 To insert : 1340947690 To insert : 1340947691 To insert : 1340947692 To insert : 1340947693 To insert : 1340947694 To insert : 1340947695 To insert : 1340947696 To insert : 1340947697 To insert : 1340947698 To insert : 1340947699 To insert : 1340947700 To insert : 1340947701 To insert : 1340947702 To insert : 1340947703 To insert : 1340947704 To insert : 1340947705 To insert : 1340947706 To insert : 1340947707 To insert : 1340947708 To insert : 1340947709 To insert : 1340947710 To insert : 1340947711 To insert : 1340947713 To insert : 1340947714 To insert : 1340947715 To insert : 1340947654 To insert : 1340947655 To insert : 1340947656 To insert : 1340947657 To insert : 1340947658 To insert : 1340947659 To insert : 1340947660 To insert : 1340947661 To insert : 1340947662 To insert : 1340947663 To insert : 1340947664 To insert : 1340947665 To insert : 1340947666 To insert : 1340947667 To insert : 1340947668 To insert : 1340947669 To insert : 1340947671 To insert : 1340947672 To insert : 1340947673 To insert : 1340947674 To insert : 1340947675 To insert : 1340947676 To insert : 1340947677 To insert : 1340947678 To insert : 1340947679 To insert : 1340947680 To insert : 1340947681 To insert : 1340947682 To insert : 1340947683 To insert : 1340947684 To insert : 1340947732 To insert : 1340947733 To insert : 1340947734 To insert : 1340947735 To insert : 1340947736 To insert : 1340947737 To insert : 1340947742 To insert : 1340947743 To insert : 1340947744 To insert : 1340947745 To insert : 1340947746 To insert : 1340947747 To insert : 1340947748 To insert : 1340947749 To insert : 1340947750 To insert : 1340947716 To insert : 1340947717 To insert : 1340947718 To insert : 1340947719 To insert : 1340947720 To insert : 1340947721 To insert : 1340947722 To insert : 1340947723 To insert : 1340947724 To insert : 1340947725 To insert : 1340947726 To insert : 1340947727 To insert : 1340947728 To insert : 1340947729 To insert : 1340947730 To insert : 1340947639 To insert : 1340947640 To insert : 1340947641 To insert : 1340947642 To insert : 1340947643 To insert : 1340947644 To insert : 1340947645 To insert : 1340947646 To insert : 1340947647 To insert : 1340947648 To insert : 1340947649 To insert : 1340947650 To insert : 1340947651 To insert : 1340947652 To insert : 1340947653 time to insert the descriptors : 26.4848895072937 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos SAVE THCL : begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 204 time used for this insertion : 0.02448749542236328 save missing photos in datou_result : time spend for datou_step_exec : 33.56386709213257 time spend to save output : 0.07645869255065918 total time spend for step 6 : 33.64032578468323 step7:ventilate_hashtags_in_portfolio Wed Mar 5 06:22:22 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 21027618 get user id for portfolio 21027618 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027618 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre','Tetrapak','pet_fonce','Carton','PET_fonce','Film_plastique','pehd','environnement','refus','flou','pet_clair','mal_croppe','PET_clair','metal','carton','Papier','papier','PEHD')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027618 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre','Tetrapak','pet_fonce','Carton','PET_fonce','Film_plastique','pehd','environnement','refus','flou','pet_clair','mal_croppe','PET_clair','metal','carton','Papier','papier','PEHD')) AND mptpi.`min_score`=0.6 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027618 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre','Tetrapak','pet_fonce','Carton','PET_fonce','Film_plastique','pehd','environnement','refus','flou','pet_clair','mal_croppe','PET_clair','metal','carton','Papier','papier','PEHD')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027618 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre','Tetrapak','pet_fonce','Carton','PET_fonce','Film_plastique','pehd','environnement','refus','flou','pet_clair','mal_croppe','PET_clair','metal','carton','Papier','papier','PEHD')) AND mptpi.`min_score`=0.6 To do lien utilise dans velours : https://www.fotonower.com/velours/21047197,21047198,21047201,21047211,21047202,21047214,21047204,21047205,21047206,21047209,21047208,21047210,21047213?tags=autre,tetrapak,pet_fonce,carton,film_plastique,pehd,environnement,refus,flou,pet_clair,mal_croppe,metal,papier&datou_id_consolidate=4235&port_consolidate=21027618 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1340722929] Looping around the photos to save general results len do output : 1 /21027618. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2634220') ('4234', '21027618', '1340722929', None, None, None, None, None, '2634220') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 2 time used for this insertion : 0.017110586166381836 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.519397497177124 time spend to save output : 0.0173947811126709 total time spend for step 7 : 2.536792278289795 step8:final Wed Mar 5 06:22:25 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1340722929: ('0.705997811091849',)} new output for save of step final : {1340722929: ('0.705997811091849',)} [1340722929] Looping around the photos to save general results len do output : 1 /1340722929.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2634220') ('4234', '21027618', '1340722929', None, None, None, None, None, '2634220') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.016225814819335938 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.116485595703125 time spend to save output : 0.016444683074951172 total time spend for step 8 : 0.13293027877807617 step9:velours_tree Wed Mar 5 06:22:25 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure list_portfolios : 21047197,21047198,21047201,21047211,21047202,21047214,21047204,21047205,21047206,21047209,21047208,21047210,21047213 photo desc type : 5561 - Retrieving photos to tag... query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21047197 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21047198 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21047201 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21047211 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21047202 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21047214 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21047204 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21047205 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21047206 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21047209 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21047208 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21047210 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21047213 ORDER BY ph.size desc - Loading descriptors... Size : 2048 len(descriptors) : 145 Compute structured hierarchical clustering... ward : AgglomerativeClustering(n_clusters=6) ward.labels_ : [3 3 0 3 0 0 2 2 0 0 0 0 4 0 1 4 1 1 1 0 4 3 0 0 4 1 2 1 1 0 4 4 4 0 0 1 0 3 1 3 5 0 0 0 0 0 5 3 3 4 3 3 0 0 1 0 2 0 4 0 5 5 4 3 3 3 4 0 0 5 3 0 0 3 0 1 3 0 0 0 0 0 0 4 3 3 0 0 0 5 4 4 0 0 0 0 0 0 3 2 0 2 2 4 2 4 3 4 0 4 2 1 2 2 3 0 4 3 3 4 4 0 0 3 3 0 0 0 4 4 2 3 4 0 5 5 3 1 0 0 0 4 0 1 0] Elapsed time: 0.0304872989654541 graph_id used : 81514 - Beta version, working pretty good on 11-5-16 ! https://www.fotonower.com/velours/21047197,21047198,21047201,21047211,21047202,21047214,21047204,21047205,21047206,21047209,21047208,21047210,21047213?tags=autre,tetrapak,pet_fonce,carton,film_plastique,pehd,environnement,refus,flou,pet_clair,mal_croppe,metal,papier&datou_id_consolidate=4235&port_consolidate=21027618&tree_id=81514 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : velours_tree we use saveGeneral [1340722929] Looping around the photos to save general results len do output : 1 /21027618Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2634220') ('4234', '21027618', '1340722929', None, None, None, None, None, '2634220') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 2 time used for this insertion : 0.01607203483581543 save_final save missing photos in datou_result : time spend for datou_step_exec : 10.444467306137085 time spend to save output : 0.016402244567871094 total time spend for step 9 : 10.460869550704956 step10:send_mail_cod Wed Mar 5 06:22:36 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 complete output_args for input 2 complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Qualipapia_P21027618_05-03-2025_06_22_36.pdf 21047179 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette210471791741152156 21047180 imagette210471801741152156 21047183 change filename to text .imagette210471831741152156 21047193 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette210471931741152156 21047184 imagette210471841741152157 21047196 imagette210471961741152157 21047187 imagette210471871741152157 21047188 imagette210471881741152157 21047191 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette210471911741152157 21047190 imagette210471901741152157 21047192 imagette210471921741152157 21047195 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette210471951741152157 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=21027618 and hashtag_type = 4230 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/21047197,21047198,21047201,21047211,21047202,21047214,21047204,21047205,21047206,21047209,21047208,21047210,21047213?tags=autre,tetrapak,pet_fonce,carton,film_plastique,pehd,environnement,refus,flou,pet_clair,mal_croppe,metal,papier&datou_id_consolidate=4235&port_consolidate=21027618&tree_id=81514 your option no_mail is active, we will not send the real mail to your client args[1340722929] : ((1340722929, -7.064070272217445, 492609224), (1340722929, 0.27422776251581044, 2107752395), '0.705997811091849') We are sending mail with results at cod@fotonower.com refus_total : 0.705997811091849 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21027618 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027618_05-03-2025_06_22_36.pdf results_Qualipapia_P21027618_05-03-2025_06_22_36.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027618_05-03-2025_06_22_36.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4234','21027618','results_Qualipapia_P21027618_05-03-2025_06_22_36.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027618_05-03-2025_06_22_36.pdf','pdf','','0.12','0.705997811091849') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1340722929] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2634220') ('4234', '21027618', '1340722929', None, None, None, None, None, '2634220') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 1 time used for this insertion : 0.01814746856689453 save_final save missing photos in datou_result : time spend for datou_step_exec : 3.9912912845611572 time spend to save output : 0.018335819244384766 total time spend for step 10 : 4.009627103805542 step11:split_time_score Wed Mar 5 06:22:40 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 3442, 'mtr_user_id': 31, 'name': 'classifieur_2camions_valcor_021122_v1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'deux_camions,camion_droite,camion_gauche,pas_de_camion', 'svm_portfolios_learning': '7659379,7659034,7657685,7657114', 'photo_hashtag_type': 4458, 'photo_desc_type': 5723, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107760533,2107760534,2107760535,2107760536'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('20', 1),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 04032025 21027618 Nombre de photos uploadées : 1 / 23040 (0%) 04032025 21027618 Nombre de photos taguées (types de déchets): 0 / 1 (0%) 04032025 21027618 Nombre de photos taguées (volume) : 0 / 1 (0%) elapsed_time : load_data_split_time_score 2.6226043701171875e-06 elapsed_time : order_list_meta_photo_and_scores 7.152557373046875e-06 ? elapsed_time : fill_and_build_computed_from_old_data 0.0001552104949951172 elapsed_time : insert_dashboard_record_day_entry 0.023076295852661133 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027616 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027617 order by id desc limit 1 Qualite : 0.705997811091849 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027618_05-03-2025_06_22_36.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027618 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027618 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027619 order by id desc limit 1 Qualite : 0.7743390429386907 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027668_05-03-2025_05_35_44.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027668 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027668 AND mptpi.`type`=4230 To do Qualite : 0.7838132502748605 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027669_05-03-2025_05_47_59.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027669 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027669 AND mptpi.`type`=4230 To do Qualite : 0.7965696585183447 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027670_05-03-2025_04_49_04.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027670 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027670 AND mptpi.`type`=4230 To do Qualite : 0.8040172653862341 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027671_05-03-2025_04_23_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027671 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027671 AND mptpi.`type`=4230 To do Qualite : 0.8661318078018976 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027672_05-03-2025_01_34_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027672 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027672 AND mptpi.`type`=4230 To do Qualite : 0.7959813288093653 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027673_05-03-2025_01_49_19.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027673 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027673 AND mptpi.`type`=4230 To do Qualite : 0.834470763448575 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027674_05-03-2025_03_48_54.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027674 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027674 AND mptpi.`type`=4230 To do Qualite : 0.851646311193362 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027675_05-03-2025_00_48_42.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027675 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027675 AND mptpi.`type`=4230 To do Qualite : 0.7966271289732663 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027676_05-03-2025_00_36_58.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027676 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027676 AND mptpi.`type`=4230 To do Qualite : 0.799150804878469 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21009642_04-03-2025_21_26_08.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21009642 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21009642 AND mptpi.`type`=4230 To do Qualite : 0.7978167118520791 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027677_04-03-2025_23_49_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027677 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027677 AND mptpi.`type`=4230 To do Qualite : 0.8168670033242211 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027678_04-03-2025_22_34_26.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027678 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027678 AND mptpi.`type`=4230 To do Qualite : 0.8002751529547594 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027679_04-03-2025_22_48_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027679 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027679 AND mptpi.`type`=4230 To do Qualite : 0.7799330629939157 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027680_04-03-2025_23_32_32.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027680 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027680 AND mptpi.`type`=4230 To do Qualite : 0.7920543799595827 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027681_04-03-2025_22_02_31.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027681 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027681 AND mptpi.`type`=4230 To do Qualite : 0.7743571987821605 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027682_04-03-2025_22_18_19.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027682 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027682 AND mptpi.`type`=4230 To do Qualite : 0.7757546150050475 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027683_04-03-2025_22_24_43.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027683 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027683 AND mptpi.`type`=4230 To do Qualite : 0.7784556948713458 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21030444_04-03-2025_23_21_47.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21030444 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21030444 AND mptpi.`type`=4230 To do Qualite : 0.7581416845186528 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21033064_05-03-2025_00_19_39.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21033064 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21033064 AND mptpi.`type`=4230 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'04032025': {'nb_upload': 1, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1340722929] Looping around the photos to save general results len do output : 1 /21027618Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2634220') ('4234', '21027618', '1340722929', None, None, None, None, None, '2634220') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 2 time used for this insertion : 0.015238046646118164 save_final save missing photos in datou_result : time spend for datou_step_exec : 3.777644157409668 time spend to save output : 0.015424489974975586 total time spend for step 11 : 3.7930686473846436 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 1 len(input) + len(total_photo_id_missing) : 1 set_done_treatment 32.82user 23.67system 2:17.18elapsed 41%CPU (0avgtext+0avgdata 2342324maxresident)k 714064inputs+13688outputs (411major+2512485minor)pagefaults 0swaps