python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 4234 -l 5 ' -s datou_current_4234 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 716307 load datou : 4234 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load THCL from format json or kwargs add thcl : 3237 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig add pdt : 5561 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4234, datou_cur_ids : ['2646451'] with mtr_portfolio_ids : ['21130892'] and first list_photo_ids : [] new path : /proc/716307/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, brightness, blur_detection, rle_unique_nms_with_priority, crop_condition, thcl, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , BFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 2 ; length of list_pids : 2 ; length of list_args : 2 time to download the photos : 0.28356409072875977 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Fri Mar 7 03:20:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10372 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-03-07 03:20:31.898759: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-03-07 03:20:31.927141: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-03-07 03:20:31.929029: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f20cc000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-03-07 03:20:31.929090: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-03-07 03:20:31.932169: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-03-07 03:20:32.057769: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1f9058d0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-03-07 03:20:32.057816: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-03-07 03:20:32.058803: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-07 03:20:32.059248: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-07 03:20:32.061881: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-07 03:20:32.064963: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-07 03:20:32.065416: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-07 03:20:32.068892: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-07 03:20:32.070658: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-07 03:20:32.077991: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-07 03:20:32.079814: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-07 03:20:32.079936: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-07 03:20:32.080862: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-07 03:20:32.080884: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-07 03:20:32.080896: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-07 03:20:32.082421: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9607 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-03-07 03:20:32.377808: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-07 03:20:32.377956: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-07 03:20:32.377985: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-07 03:20:32.378009: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-07 03:20:32.378034: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-07 03:20:32.378057: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-07 03:20:32.378080: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-07 03:20:32.378102: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-07 03:20:32.380403: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-07 03:20:32.381951: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-07 03:20:32.382039: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-07 03:20:32.382062: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-07 03:20:32.382084: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-07 03:20:32.382104: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-07 03:20:32.382124: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-07 03:20:32.382145: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-07 03:20:32.382166: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-07 03:20:32.386237: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-07 03:20:32.386294: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-07 03:20:32.386309: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-07 03:20:32.386319: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-07 03:20:32.387940: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9607 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-03-07 03:20:40.762646: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-07 03:20:40.994261: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 2 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -119.46953 max: 145.67812 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 9.942054748535156e-05 nb_pixel_total : 454 time to create 1 rle with old method : 0.0022678375244140625 length of segment : 39 time for calcul the mask position with numpy : 5.245208740234375e-05 nb_pixel_total : 117 time to create 1 rle with old method : 0.0003006458282470703 length of segment : 15 time for calcul the mask position with numpy : 5.245208740234375e-05 nb_pixel_total : 460 time to create 1 rle with old method : 0.005110263824462891 length of segment : 40 time for calcul the mask position with numpy : 6.842613220214844e-05 nb_pixel_total : 644 time to create 1 rle with old method : 0.0013074874877929688 length of segment : 48 time for calcul the mask position with numpy : 8.749961853027344e-05 nb_pixel_total : 713 time to create 1 rle with old method : 0.0014774799346923828 length of segment : 51 time for calcul the mask position with numpy : 8.845329284667969e-05 nb_pixel_total : 1677 time to create 1 rle with old method : 0.0027573108673095703 length of segment : 53 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -116.91484 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 4.6253204345703125e-05 nb_pixel_total : 599 time to create 1 rle with old method : 0.0008220672607421875 length of segment : 74 time for calcul the mask position with numpy : 8.535385131835938e-05 nb_pixel_total : 2457 time to create 1 rle with old method : 0.0032966136932373047 length of segment : 50 time for calcul the mask position with numpy : 5.125999450683594e-05 nb_pixel_total : 975 time to create 1 rle with old method : 0.0014123916625976562 length of segment : 61 time for calcul the mask position with numpy : 3.814697265625e-05 nb_pixel_total : 373 time to create 1 rle with old method : 0.0005133152008056641 length of segment : 46 time for calcul the mask position with numpy : 5.3882598876953125e-05 nb_pixel_total : 1044 time to create 1 rle with old method : 0.0013625621795654297 length of segment : 51 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -113.23906 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 6.890296936035156e-05 nb_pixel_total : 1842 time to create 1 rle with old method : 0.002928495407104492 length of segment : 87 time for calcul the mask position with numpy : 5.364418029785156e-05 nb_pixel_total : 443 time to create 1 rle with old method : 0.0008897781372070312 length of segment : 11 time for calcul the mask position with numpy : 0.00012612342834472656 nb_pixel_total : 4486 time to create 1 rle with old method : 0.0062677860260009766 length of segment : 65 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.12578 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 4.172325134277344e-05 nb_pixel_total : 383 time to create 1 rle with old method : 0.0005619525909423828 length of segment : 17 time for calcul the mask position with numpy : 3.933906555175781e-05 nb_pixel_total : 754 time to create 1 rle with old method : 0.0010175704956054688 length of segment : 36 time for calcul the mask position with numpy : 5.412101745605469e-05 nb_pixel_total : 422 time to create 1 rle with old method : 0.0006463527679443359 length of segment : 26 time for calcul the mask position with numpy : 4.410743713378906e-05 nb_pixel_total : 432 time to create 1 rle with old method : 0.0007443428039550781 length of segment : 26 time for calcul the mask position with numpy : 3.6716461181640625e-05 nb_pixel_total : 358 time to create 1 rle with old method : 0.0005414485931396484 length of segment : 30 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 470 time to create 1 rle with old method : 0.0007998943328857422 length of segment : 29 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.07656 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 11 time for calcul the mask position with numpy : 6.4849853515625e-05 nb_pixel_total : 799 time to create 1 rle with old method : 0.001528024673461914 length of segment : 80 time for calcul the mask position with numpy : 5.8650970458984375e-05 nb_pixel_total : 663 time to create 1 rle with old method : 0.001377105712890625 length of segment : 31 time for calcul the mask position with numpy : 5.6743621826171875e-05 nb_pixel_total : 541 time to create 1 rle with old method : 0.001272439956665039 length of segment : 18 time for calcul the mask position with numpy : 9.489059448242188e-05 nb_pixel_total : 1155 time to create 1 rle with old method : 0.002555370330810547 length of segment : 86 time for calcul the mask position with numpy : 0.00014829635620117188 nb_pixel_total : 838 time to create 1 rle with old method : 0.0018694400787353516 length of segment : 33 time for calcul the mask position with numpy : 4.1484832763671875e-05 nb_pixel_total : 555 time to create 1 rle with old method : 0.0008392333984375 length of segment : 18 time for calcul the mask position with numpy : 4.649162292480469e-05 nb_pixel_total : 560 time to create 1 rle with old method : 0.0008540153503417969 length of segment : 62 time for calcul the mask position with numpy : 9.703636169433594e-05 nb_pixel_total : 887 time to create 1 rle with old method : 0.0012133121490478516 length of segment : 61 time for calcul the mask position with numpy : 6.0558319091796875e-05 nb_pixel_total : 1105 time to create 1 rle with old method : 0.001580953598022461 length of segment : 83 time for calcul the mask position with numpy : 0.0006928443908691406 nb_pixel_total : 48287 time to create 1 rle with old method : 0.060898780822753906 length of segment : 356 time for calcul the mask position with numpy : 6.604194641113281e-05 nb_pixel_total : 83 time to create 1 rle with old method : 0.00017261505126953125 length of segment : 10 Processing 1 images image shape: (400, 400, 3) min: 21.00000 max: 230.00000 molded_images shape: (1, 640, 640, 3) min: -95.00859 max: 106.41250 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -107.11406 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 10 time for calcul the mask position with numpy : 6.4849853515625e-05 nb_pixel_total : 1186 time to create 1 rle with old method : 0.0017824172973632812 length of segment : 29 time for calcul the mask position with numpy : 4.1484832763671875e-05 nb_pixel_total : 240 time to create 1 rle with old method : 0.0004851818084716797 length of segment : 14 time for calcul the mask position with numpy : 4.00543212890625e-05 nb_pixel_total : 297 time to create 1 rle with old method : 0.0004436969757080078 length of segment : 21 time for calcul the mask position with numpy : 0.00018739700317382812 nb_pixel_total : 9072 time to create 1 rle with old method : 0.013773918151855469 length of segment : 134 time for calcul the mask position with numpy : 5.2928924560546875e-05 nb_pixel_total : 315 time to create 1 rle with old method : 0.0004887580871582031 length of segment : 19 time for calcul the mask position with numpy : 4.0531158447265625e-05 nb_pixel_total : 291 time to create 1 rle with old method : 0.0004317760467529297 length of segment : 21 time for calcul the mask position with numpy : 3.552436828613281e-05 nb_pixel_total : 89 time to create 1 rle with old method : 0.00016260147094726562 length of segment : 11 time for calcul the mask position with numpy : 8.96453857421875e-05 nb_pixel_total : 3072 time to create 1 rle with old method : 0.005042314529418945 length of segment : 43 time for calcul the mask position with numpy : 4.6253204345703125e-05 nb_pixel_total : 143 time to create 1 rle with old method : 0.0002541542053222656 length of segment : 11 time for calcul the mask position with numpy : 6.580352783203125e-05 nb_pixel_total : 1276 time to create 1 rle with old method : 0.0017547607421875 length of segment : 52 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -119.17656 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 7.033348083496094e-05 nb_pixel_total : 318 time to create 1 rle with old method : 0.0007650852203369141 length of segment : 29 time for calcul the mask position with numpy : 0.0004496574401855469 nb_pixel_total : 15406 time to create 1 rle with old method : 0.025270462036132812 length of segment : 270 time for calcul the mask position with numpy : 5.14984130859375e-05 nb_pixel_total : 151 time to create 1 rle with old method : 0.0002334117889404297 length of segment : 14 time for calcul the mask position with numpy : 2.7894973754882812e-05 nb_pixel_total : 27 time to create 1 rle with old method : 6.270408630371094e-05 length of segment : 5 time for calcul the mask position with numpy : 3.337860107421875e-05 nb_pixel_total : 157 time to create 1 rle with old method : 0.0002384185791015625 length of segment : 14 time for calcul the mask position with numpy : 3.0040740966796875e-05 nb_pixel_total : 100 time to create 1 rle with old method : 0.0001919269561767578 length of segment : 8 time for calcul the mask position with numpy : 3.8623809814453125e-05 nb_pixel_total : 375 time to create 1 rle with old method : 0.0005819797515869141 length of segment : 23 time for calcul the mask position with numpy : 3.2901763916015625e-05 nb_pixel_total : 193 time to create 1 rle with old method : 0.00029850006103515625 length of segment : 17 time for calcul the mask position with numpy : 0.00011706352233886719 nb_pixel_total : 2404 time to create 1 rle with old method : 0.0032541751861572266 length of segment : 140 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -122.80156 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 5.125999450683594e-05 nb_pixel_total : 709 time to create 1 rle with old method : 0.0010297298431396484 length of segment : 28 time for calcul the mask position with numpy : 6.413459777832031e-05 nb_pixel_total : 1336 time to create 1 rle with old method : 0.0019528865814208984 length of segment : 43 time for calcul the mask position with numpy : 3.886222839355469e-05 nb_pixel_total : 217 time to create 1 rle with old method : 0.0003371238708496094 length of segment : 22 time for calcul the mask position with numpy : 3.981590270996094e-05 nb_pixel_total : 378 time to create 1 rle with old method : 0.0005781650543212891 length of segment : 21 time for calcul the mask position with numpy : 6.532669067382812e-05 nb_pixel_total : 1709 time to create 1 rle with old method : 0.0023293495178222656 length of segment : 46 time for calcul the mask position with numpy : 5.626678466796875e-05 nb_pixel_total : 725 time to create 1 rle with old method : 0.0009489059448242188 length of segment : 36 time for calcul the mask position with numpy : 8.96453857421875e-05 nb_pixel_total : 3067 time to create 1 rle with old method : 0.004115581512451172 length of segment : 81 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.42031 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 10 time for calcul the mask position with numpy : 4.76837158203125e-05 nb_pixel_total : 619 time to create 1 rle with old method : 0.0008549690246582031 length of segment : 42 time for calcul the mask position with numpy : 7.2479248046875e-05 nb_pixel_total : 674 time to create 1 rle with old method : 0.0012967586517333984 length of segment : 78 time for calcul the mask position with numpy : 8.0108642578125e-05 nb_pixel_total : 1519 time to create 1 rle with old method : 0.0019485950469970703 length of segment : 74 time for calcul the mask position with numpy : 0.0001652240753173828 nb_pixel_total : 3394 time to create 1 rle with old method : 0.004437923431396484 length of segment : 173 time for calcul the mask position with numpy : 9.226799011230469e-05 nb_pixel_total : 684 time to create 1 rle with old method : 0.0009829998016357422 length of segment : 73 time for calcul the mask position with numpy : 0.0001976490020751953 nb_pixel_total : 503 time to create 1 rle with old method : 0.0007212162017822266 length of segment : 55 time for calcul the mask position with numpy : 0.00025844573974609375 nb_pixel_total : 4532 time to create 1 rle with old method : 0.005765199661254883 length of segment : 219 time for calcul the mask position with numpy : 4.935264587402344e-05 nb_pixel_total : 572 time to create 1 rle with old method : 0.0007638931274414062 length of segment : 39 time for calcul the mask position with numpy : 6.079673767089844e-05 nb_pixel_total : 1305 time to create 1 rle with old method : 0.0018398761749267578 length of segment : 46 time for calcul the mask position with numpy : 0.00016999244689941406 nb_pixel_total : 4195 time to create 1 rle with old method : 0.005465030670166016 length of segment : 223 Processing 1 images image shape: (280, 400, 3) min: 8.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 130.77969 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 7.43865966796875e-05 nb_pixel_total : 620 time to create 1 rle with old method : 0.0009055137634277344 length of segment : 71 time for calcul the mask position with numpy : 0.0013110637664794922 nb_pixel_total : 105615 time to create 1 rle with old method : 0.1302509307861328 length of segment : 280 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 13 time for calcul the mask position with numpy : 7.271766662597656e-05 nb_pixel_total : 782 time to create 1 rle with old method : 0.0010488033294677734 length of segment : 60 time for calcul the mask position with numpy : 5.3882598876953125e-05 nb_pixel_total : 382 time to create 1 rle with old method : 0.0011713504791259766 length of segment : 32 time for calcul the mask position with numpy : 5.245208740234375e-05 nb_pixel_total : 605 time to create 1 rle with old method : 0.0008835792541503906 length of segment : 56 time for calcul the mask position with numpy : 0.00014209747314453125 nb_pixel_total : 4672 time to create 1 rle with old method : 0.006059169769287109 length of segment : 168 time for calcul the mask position with numpy : 0.00017261505126953125 nb_pixel_total : 4971 time to create 1 rle with old method : 0.00650787353515625 length of segment : 86 time for calcul the mask position with numpy : 4.887580871582031e-05 nb_pixel_total : 183 time to create 1 rle with old method : 0.0002963542938232422 length of segment : 14 time for calcul the mask position with numpy : 7.033348083496094e-05 nb_pixel_total : 2027 time to create 1 rle with old method : 0.0028100013732910156 length of segment : 32 time for calcul the mask position with numpy : 4.553794860839844e-05 nb_pixel_total : 359 time to create 1 rle with old method : 0.0005540847778320312 length of segment : 22 time for calcul the mask position with numpy : 7.486343383789062e-05 nb_pixel_total : 370 time to create 1 rle with old method : 0.0008039474487304688 length of segment : 30 time for calcul the mask position with numpy : 4.696846008300781e-05 nb_pixel_total : 231 time to create 1 rle with old method : 0.00036597251892089844 length of segment : 31 time for calcul the mask position with numpy : 5.888938903808594e-05 nb_pixel_total : 1636 time to create 1 rle with old method : 0.0023262500762939453 length of segment : 31 time for calcul the mask position with numpy : 6.580352783203125e-05 nb_pixel_total : 1563 time to create 1 rle with old method : 0.0020716190338134766 length of segment : 41 time for calcul the mask position with numpy : 6.985664367675781e-05 nb_pixel_total : 1937 time to create 1 rle with old method : 0.0026328563690185547 length of segment : 31 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 27 time for calcul the mask position with numpy : 9.989738464355469e-05 nb_pixel_total : 728 time to create 1 rle with old method : 0.0010333061218261719 length of segment : 37 time for calcul the mask position with numpy : 4.601478576660156e-05 nb_pixel_total : 372 time to create 1 rle with old method : 0.0005486011505126953 length of segment : 19 time for calcul the mask position with numpy : 3.886222839355469e-05 nb_pixel_total : 177 time to create 1 rle with old method : 0.00027632713317871094 length of segment : 18 time for calcul the mask position with numpy : 5.1021575927734375e-05 nb_pixel_total : 695 time to create 1 rle with old method : 0.0009989738464355469 length of segment : 27 time for calcul the mask position with numpy : 3.9577484130859375e-05 nb_pixel_total : 212 time to create 1 rle with old method : 0.00033164024353027344 length of segment : 17 time for calcul the mask position with numpy : 3.337860107421875e-05 nb_pixel_total : 62 time to create 1 rle with old method : 0.00013375282287597656 length of segment : 8 time for calcul the mask position with numpy : 3.457069396972656e-05 nb_pixel_total : 175 time to create 1 rle with old method : 0.00027108192443847656 length of segment : 13 time for calcul the mask position with numpy : 4.172325134277344e-05 nb_pixel_total : 428 time to create 1 rle with old method : 0.0006113052368164062 length of segment : 28 time for calcul the mask position with numpy : 4.029273986816406e-05 nb_pixel_total : 308 time to create 1 rle with old method : 0.0004582405090332031 length of segment : 22 time for calcul the mask position with numpy : 3.8623809814453125e-05 nb_pixel_total : 217 time to create 1 rle with old method : 0.0003495216369628906 length of segment : 17 time for calcul the mask position with numpy : 4.076957702636719e-05 nb_pixel_total : 212 time to create 1 rle with old method : 0.0003669261932373047 length of segment : 21 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 355 time to create 1 rle with old method : 0.0005273818969726562 length of segment : 26 time for calcul the mask position with numpy : 6.937980651855469e-05 nb_pixel_total : 990 time to create 1 rle with old method : 0.0014123916625976562 length of segment : 35 time for calcul the mask position with numpy : 4.1961669921875e-05 nb_pixel_total : 193 time to create 1 rle with old method : 0.0002987384796142578 length of segment : 19 time for calcul the mask position with numpy : 3.3855438232421875e-05 nb_pixel_total : 73 time to create 1 rle with old method : 0.0001327991485595703 length of segment : 11 time for calcul the mask position with numpy : 5.269050598144531e-05 nb_pixel_total : 1248 time to create 1 rle with old method : 0.0017476081848144531 length of segment : 33 time for calcul the mask position with numpy : 4.267692565917969e-05 nb_pixel_total : 296 time to create 1 rle with old method : 0.0004563331604003906 length of segment : 17 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 411 time to create 1 rle with old method : 0.0005905628204345703 length of segment : 32 time for calcul the mask position with numpy : 3.600120544433594e-05 nb_pixel_total : 169 time to create 1 rle with old method : 0.0002894401550292969 length of segment : 15 time for calcul the mask position with numpy : 3.814697265625e-05 nb_pixel_total : 209 time to create 1 rle with old method : 0.000347137451171875 length of segment : 20 time for calcul the mask position with numpy : 3.743171691894531e-05 nb_pixel_total : 160 time to create 1 rle with old method : 0.00024819374084472656 length of segment : 23 time for calcul the mask position with numpy : 3.2901763916015625e-05 nb_pixel_total : 97 time to create 1 rle with old method : 0.0001659393310546875 length of segment : 12 time for calcul the mask position with numpy : 4.57763671875e-05 nb_pixel_total : 214 time to create 1 rle with old method : 0.0003361701965332031 length of segment : 16 time for calcul the mask position with numpy : 3.4809112548828125e-05 nb_pixel_total : 159 time to create 1 rle with old method : 0.000270843505859375 length of segment : 14 time for calcul the mask position with numpy : 6.771087646484375e-05 nb_pixel_total : 837 time to create 1 rle with old method : 0.0012149810791015625 length of segment : 58 time for calcul the mask position with numpy : 5.3882598876953125e-05 nb_pixel_total : 530 time to create 1 rle with old method : 0.0007989406585693359 length of segment : 25 time for calcul the mask position with numpy : 4.863739013671875e-05 nb_pixel_total : 248 time to create 1 rle with old method : 0.0003745555877685547 length of segment : 20 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.82656 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 12 time for calcul the mask position with numpy : 9.34600830078125e-05 nb_pixel_total : 952 time to create 1 rle with old method : 0.0017657279968261719 length of segment : 52 time for calcul the mask position with numpy : 6.270408630371094e-05 nb_pixel_total : 492 time to create 1 rle with old method : 0.0009672641754150391 length of segment : 27 time for calcul the mask position with numpy : 4.673004150390625e-05 nb_pixel_total : 144 time to create 1 rle with old method : 0.00033020973205566406 length of segment : 13 time for calcul the mask position with numpy : 5.8650970458984375e-05 nb_pixel_total : 1004 time to create 1 rle with old method : 0.0018928050994873047 length of segment : 42 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 470 time to create 1 rle with old method : 0.0007524490356445312 length of segment : 28 time for calcul the mask position with numpy : 6.341934204101562e-05 nb_pixel_total : 935 time to create 1 rle with old method : 0.0013794898986816406 length of segment : 73 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 212 time to create 1 rle with old method : 0.0003361701965332031 length of segment : 16 time for calcul the mask position with numpy : 4.363059997558594e-05 nb_pixel_total : 636 time to create 1 rle with old method : 0.0008366107940673828 length of segment : 43 time for calcul the mask position with numpy : 3.9577484130859375e-05 nb_pixel_total : 394 time to create 1 rle with old method : 0.0006642341613769531 length of segment : 29 time for calcul the mask position with numpy : 3.814697265625e-05 nb_pixel_total : 320 time to create 1 rle with old method : 0.0005488395690917969 length of segment : 20 time for calcul the mask position with numpy : 4.6253204345703125e-05 nb_pixel_total : 631 time to create 1 rle with old method : 0.0008368492126464844 length of segment : 47 time for calcul the mask position with numpy : 0.0001964569091796875 nb_pixel_total : 8450 time to create 1 rle with old method : 0.010572433471679688 length of segment : 140 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 141.03750 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 length of segment : 0 time for calcul the mask position with numpy : 3.504753112792969e-05 nb_pixel_total : 24 time to create 1 rle with old method : 8.678436279296875e-05 length of segment : 5 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -121.93828 max: 145.44375 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 7.510185241699219e-05 nb_pixel_total : 705 time to create 1 rle with old method : 0.0014493465423583984 length of segment : 43 time for calcul the mask position with numpy : 5.650520324707031e-05 nb_pixel_total : 109 time to create 1 rle with old method : 0.0002760887145996094 length of segment : 14 time for calcul the mask position with numpy : 7.915496826171875e-05 nb_pixel_total : 707 time to create 1 rle with old method : 0.001519918441772461 length of segment : 47 time for calcul the mask position with numpy : 8.0108642578125e-05 nb_pixel_total : 831 time to create 1 rle with old method : 0.001421213150024414 length of segment : 64 time for calcul the mask position with numpy : 7.224082946777344e-05 nb_pixel_total : 2040 time to create 1 rle with old method : 0.002863168716430664 length of segment : 49 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.03984 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 4.315376281738281e-05 nb_pixel_total : 449 time to create 1 rle with old method : 0.0006375312805175781 length of segment : 42 time for calcul the mask position with numpy : 6.747245788574219e-05 nb_pixel_total : 2477 time to create 1 rle with old method : 0.0033118724822998047 length of segment : 52 time for calcul the mask position with numpy : 4.673004150390625e-05 nb_pixel_total : 494 time to create 1 rle with old method : 0.0008950233459472656 length of segment : 68 time for calcul the mask position with numpy : 5.269050598144531e-05 nb_pixel_total : 1146 time to create 1 rle with old method : 0.001619577407836914 length of segment : 32 time for calcul the mask position with numpy : 4.482269287109375e-05 nb_pixel_total : 925 time to create 1 rle with old method : 0.0012919902801513672 length of segment : 33 time for calcul the mask position with numpy : 4.482269287109375e-05 nb_pixel_total : 891 time to create 1 rle with old method : 0.0012793540954589844 length of segment : 33 time for calcul the mask position with numpy : 7.82012939453125e-05 nb_pixel_total : 1077 time to create 1 rle with old method : 0.0014820098876953125 length of segment : 54 time for calcul the mask position with numpy : 4.2438507080078125e-05 nb_pixel_total : 853 time to create 1 rle with old method : 0.001209259033203125 length of segment : 38 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.89141 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 7.033348083496094e-05 nb_pixel_total : 2109 time to create 1 rle with old method : 0.002766847610473633 length of segment : 94 time for calcul the mask position with numpy : 4.410743713378906e-05 nb_pixel_total : 407 time to create 1 rle with old method : 0.0008082389831542969 length of segment : 8 time for calcul the mask position with numpy : 3.075599670410156e-05 nb_pixel_total : 121 time to create 1 rle with old method : 0.00028252601623535156 length of segment : 6 time for calcul the mask position with numpy : 0.00010609626770019531 nb_pixel_total : 5279 time to create 1 rle with old method : 0.006983518600463867 length of segment : 81 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -122.68438 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 4.887580871582031e-05 nb_pixel_total : 420 time to create 1 rle with old method : 0.0007448196411132812 length of segment : 19 time for calcul the mask position with numpy : 4.00543212890625e-05 nb_pixel_total : 724 time to create 1 rle with old method : 0.0009791851043701172 length of segment : 35 time for calcul the mask position with numpy : 3.4332275390625e-05 nb_pixel_total : 404 time to create 1 rle with old method : 0.0005891323089599609 length of segment : 23 time for calcul the mask position with numpy : 4.076957702636719e-05 nb_pixel_total : 485 time to create 1 rle with old method : 0.0008211135864257812 length of segment : 29 time for calcul the mask position with numpy : 0.00012350082397460938 nb_pixel_total : 6286 time to create 1 rle with old method : 0.00801396369934082 length of segment : 123 time for calcul the mask position with numpy : 6.198883056640625e-05 nb_pixel_total : 546 time to create 1 rle with old method : 0.0008976459503173828 length of segment : 33 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 149.69375 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 4.792213439941406e-05 nb_pixel_total : 754 time to create 1 rle with old method : 0.001016855239868164 length of segment : 36 time for calcul the mask position with numpy : 4.839897155761719e-05 nb_pixel_total : 1055 time to create 1 rle with old method : 0.0013818740844726562 length of segment : 117 time for calcul the mask position with numpy : 6.031990051269531e-05 nb_pixel_total : 510 time to create 1 rle with old method : 0.0008003711700439453 length of segment : 16 time for calcul the mask position with numpy : 5.1975250244140625e-05 nb_pixel_total : 753 time to create 1 rle with old method : 0.0010797977447509766 length of segment : 41 time for calcul the mask position with numpy : 0.00014138221740722656 nb_pixel_total : 4007 time to create 1 rle with old method : 0.006712913513183594 length of segment : 55 time for calcul the mask position with numpy : 0.0006556510925292969 nb_pixel_total : 49549 time to create 1 rle with old method : 0.05979800224304199 length of segment : 284 time for calcul the mask position with numpy : 9.34600830078125e-05 nb_pixel_total : 830 time to create 1 rle with old method : 0.001102447509765625 length of segment : 62 time for calcul the mask position with numpy : 4.410743713378906e-05 nb_pixel_total : 526 time to create 1 rle with old method : 0.0007803440093994141 length of segment : 17 time for calcul the mask position with numpy : 0.0010387897491455078 nb_pixel_total : 66066 time to create 1 rle with old method : 0.07938027381896973 length of segment : 372 Processing 1 images image shape: (400, 400, 3) min: 19.00000 max: 240.00000 molded_images shape: (1, 640, 640, 3) min: -98.05547 max: 123.38906 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -101.67266 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 12 time for calcul the mask position with numpy : 6.747245788574219e-05 nb_pixel_total : 1206 time to create 1 rle with old method : 0.0019352436065673828 length of segment : 29 time for calcul the mask position with numpy : 4.3392181396484375e-05 nb_pixel_total : 288 time to create 1 rle with old method : 0.0004286766052246094 length of segment : 22 time for calcul the mask position with numpy : 3.6716461181640625e-05 nb_pixel_total : 268 time to create 1 rle with old method : 0.00043201446533203125 length of segment : 16 time for calcul the mask position with numpy : 4.172325134277344e-05 nb_pixel_total : 327 time to create 1 rle with old method : 0.0005095005035400391 length of segment : 19 time for calcul the mask position with numpy : 3.838539123535156e-05 nb_pixel_total : 336 time to create 1 rle with old method : 0.0005042552947998047 length of segment : 22 time for calcul the mask position with numpy : 3.695487976074219e-05 nb_pixel_total : 285 time to create 1 rle with old method : 0.0004353523254394531 length of segment : 23 time for calcul the mask position with numpy : 0.0001888275146484375 nb_pixel_total : 8075 time to create 1 rle with old method : 0.010427236557006836 length of segment : 138 time for calcul the mask position with numpy : 4.863739013671875e-05 nb_pixel_total : 75 time to create 1 rle with old method : 0.00014352798461914062 length of segment : 11 time for calcul the mask position with numpy : 3.3855438232421875e-05 nb_pixel_total : 109 time to create 1 rle with old method : 0.00020074844360351562 length of segment : 10 time for calcul the mask position with numpy : 0.0003101825714111328 nb_pixel_total : 17194 time to create 1 rle with old method : 0.02196526527404785 length of segment : 124 time for calcul the mask position with numpy : 5.8650970458984375e-05 nb_pixel_total : 425 time to create 1 rle with old method : 0.0006513595581054688 length of segment : 19 time for calcul the mask position with numpy : 3.719329833984375e-05 nb_pixel_total : 98 time to create 1 rle with old method : 0.0001766681671142578 length of segment : 12 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -119.18047 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 0.0002779960632324219 nb_pixel_total : 9500 time to create 1 rle with old method : 0.014931917190551758 length of segment : 341 time for calcul the mask position with numpy : 5.53131103515625e-05 nb_pixel_total : 29 time to create 1 rle with old method : 7.700920104980469e-05 length of segment : 6 time for calcul the mask position with numpy : 6.198883056640625e-05 nb_pixel_total : 599 time to create 1 rle with old method : 0.0008018016815185547 length of segment : 40 time for calcul the mask position with numpy : 6.222724914550781e-05 nb_pixel_total : 1171 time to create 1 rle with old method : 0.0015716552734375 length of segment : 50 time for calcul the mask position with numpy : 4.1484832763671875e-05 nb_pixel_total : 147 time to create 1 rle with old method : 0.00023674964904785156 length of segment : 15 time for calcul the mask position with numpy : 4.982948303222656e-05 nb_pixel_total : 549 time to create 1 rle with old method : 0.0010213851928710938 length of segment : 12 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -122.20000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 0.00014972686767578125 nb_pixel_total : 236 time to create 1 rle with old method : 0.0005185604095458984 length of segment : 23 time for calcul the mask position with numpy : 7.677078247070312e-05 nb_pixel_total : 708 time to create 1 rle with old method : 0.001338958740234375 length of segment : 27 time for calcul the mask position with numpy : 7.033348083496094e-05 nb_pixel_total : 1311 time to create 1 rle with old method : 0.0022258758544921875 length of segment : 40 time for calcul the mask position with numpy : 0.00011920928955078125 nb_pixel_total : 2681 time to create 1 rle with old method : 0.004610776901245117 length of segment : 83 time for calcul the mask position with numpy : 4.601478576660156e-05 nb_pixel_total : 220 time to create 1 rle with old method : 0.0003380775451660156 length of segment : 15 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 749 time to create 1 rle with old method : 0.001018524169921875 length of segment : 37 time for calcul the mask position with numpy : 3.3855438232421875e-05 nb_pixel_total : 369 time to create 1 rle with old method : 0.0005338191986083984 length of segment : 22 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.60000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 11 time for calcul the mask position with numpy : 9.942054748535156e-05 nb_pixel_total : 924 time to create 1 rle with old method : 0.0014009475708007812 length of segment : 120 time for calcul the mask position with numpy : 5.841255187988281e-05 nb_pixel_total : 331 time to create 1 rle with old method : 0.0005426406860351562 length of segment : 27 time for calcul the mask position with numpy : 5.555152893066406e-05 nb_pixel_total : 517 time to create 1 rle with old method : 0.0007586479187011719 length of segment : 37 time for calcul the mask position with numpy : 0.0002346038818359375 nb_pixel_total : 4055 time to create 1 rle with old method : 0.005228519439697266 length of segment : 190 time for calcul the mask position with numpy : 0.00023674964904785156 nb_pixel_total : 533 time to create 1 rle with old method : 0.000762939453125 length of segment : 63 time for calcul the mask position with numpy : 0.00013637542724609375 nb_pixel_total : 1048 time to create 1 rle with old method : 0.0014705657958984375 length of segment : 44 time for calcul the mask position with numpy : 0.00010657310485839844 nb_pixel_total : 1509 time to create 1 rle with old method : 0.001990079879760742 length of segment : 74 time for calcul the mask position with numpy : 6.4849853515625e-05 nb_pixel_total : 1033 time to create 1 rle with old method : 0.0014309883117675781 length of segment : 46 time for calcul the mask position with numpy : 0.00024008750915527344 nb_pixel_total : 4874 time to create 1 rle with old method : 0.006173610687255859 length of segment : 214 time for calcul the mask position with numpy : 0.00030231475830078125 nb_pixel_total : 4948 time to create 1 rle with old method : 0.008566141128540039 length of segment : 223 time for calcul the mask position with numpy : 8.797645568847656e-05 nb_pixel_total : 597 time to create 1 rle with old method : 0.001249551773071289 length of segment : 64 Processing 1 images image shape: (280, 400, 3) min: 12.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 127.02969 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 6.890296936035156e-05 nb_pixel_total : 574 time to create 1 rle with old method : 0.0011074542999267578 length of segment : 75 time for calcul the mask position with numpy : 0.0012521743774414062 nb_pixel_total : 105772 time to create 1 rle with old method : 0.13187599182128906 length of segment : 280 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 11 time for calcul the mask position with numpy : 0.00011539459228515625 nb_pixel_total : 5070 time to create 1 rle with old method : 0.006264925003051758 length of segment : 85 time for calcul the mask position with numpy : 4.839897155761719e-05 nb_pixel_total : 368 time to create 1 rle with old method : 0.0005285739898681641 length of segment : 30 time for calcul the mask position with numpy : 4.029273986816406e-05 nb_pixel_total : 453 time to create 1 rle with old method : 0.0006239414215087891 length of segment : 35 time for calcul the mask position with numpy : 8.153915405273438e-05 nb_pixel_total : 3141 time to create 1 rle with old method : 0.0041615962982177734 length of segment : 63 time for calcul the mask position with numpy : 3.790855407714844e-05 nb_pixel_total : 325 time to create 1 rle with old method : 0.0004940032958984375 length of segment : 21 time for calcul the mask position with numpy : 3.504753112792969e-05 nb_pixel_total : 113 time to create 1 rle with old method : 0.00020956993103027344 length of segment : 23 time for calcul the mask position with numpy : 5.221366882324219e-05 nb_pixel_total : 1624 time to create 1 rle with old method : 0.0022287368774414062 length of segment : 41 time for calcul the mask position with numpy : 3.457069396972656e-05 nb_pixel_total : 188 time to create 1 rle with old method : 0.0002913475036621094 length of segment : 15 time for calcul the mask position with numpy : 6.771087646484375e-05 nb_pixel_total : 1879 time to create 1 rle with old method : 0.0024802684783935547 length of segment : 58 time for calcul the mask position with numpy : 3.719329833984375e-05 nb_pixel_total : 477 time to create 1 rle with old method : 0.0006575584411621094 length of segment : 29 time for calcul the mask position with numpy : 4.673004150390625e-05 nb_pixel_total : 1442 time to create 1 rle with old method : 0.001889944076538086 length of segment : 39 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 33 time for calcul the mask position with numpy : 7.724761962890625e-05 nb_pixel_total : 774 time to create 1 rle with old method : 0.0014872550964355469 length of segment : 50 time for calcul the mask position with numpy : 4.1961669921875e-05 nb_pixel_total : 178 time to create 1 rle with old method : 0.0003905296325683594 length of segment : 19 time for calcul the mask position with numpy : 4.553794860839844e-05 nb_pixel_total : 400 time to create 1 rle with old method : 0.0007967948913574219 length of segment : 27 time for calcul the mask position with numpy : 4.076957702636719e-05 nb_pixel_total : 221 time to create 1 rle with old method : 0.0004589557647705078 length of segment : 18 time for calcul the mask position with numpy : 3.600120544433594e-05 nb_pixel_total : 53 time to create 1 rle with old method : 0.00012993812561035156 length of segment : 8 time for calcul the mask position with numpy : 4.029273986816406e-05 nb_pixel_total : 221 time to create 1 rle with old method : 0.00044345855712890625 length of segment : 21 time for calcul the mask position with numpy : 3.9577484130859375e-05 nb_pixel_total : 166 time to create 1 rle with old method : 0.00036454200744628906 length of segment : 13 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 266 time to create 1 rle with old method : 0.0005176067352294922 length of segment : 25 time for calcul the mask position with numpy : 6.103515625e-05 nb_pixel_total : 1276 time to create 1 rle with old method : 0.002372264862060547 length of segment : 33 time for calcul the mask position with numpy : 4.076957702636719e-05 nb_pixel_total : 201 time to create 1 rle with old method : 0.0004279613494873047 length of segment : 16 time for calcul the mask position with numpy : 3.910064697265625e-05 nb_pixel_total : 181 time to create 1 rle with old method : 0.00035762786865234375 length of segment : 19 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 423 time to create 1 rle with old method : 0.0008165836334228516 length of segment : 28 time for calcul the mask position with numpy : 4.220008850097656e-05 nb_pixel_total : 704 time to create 1 rle with old method : 0.0010035037994384766 length of segment : 27 time for calcul the mask position with numpy : 3.218650817871094e-05 nb_pixel_total : 214 time to create 1 rle with old method : 0.0003254413604736328 length of segment : 18 time for calcul the mask position with numpy : 3.0517578125e-05 nb_pixel_total : 94 time to create 1 rle with old method : 0.00015878677368164062 length of segment : 12 time for calcul the mask position with numpy : 2.8848648071289062e-05 nb_pixel_total : 89 time to create 1 rle with old method : 0.0001468658447265625 length of segment : 12 time for calcul the mask position with numpy : 3.814697265625e-05 nb_pixel_total : 570 time to create 1 rle with old method : 0.0007960796356201172 length of segment : 25 time for calcul the mask position with numpy : 3.0279159545898438e-05 nb_pixel_total : 145 time to create 1 rle with old method : 0.00023126602172851562 length of segment : 15 time for calcul the mask position with numpy : 3.3855438232421875e-05 nb_pixel_total : 305 time to create 1 rle with old method : 0.0004711151123046875 length of segment : 22 time for calcul the mask position with numpy : 3.0994415283203125e-05 nb_pixel_total : 194 time to create 1 rle with old method : 0.0002906322479248047 length of segment : 15 time for calcul the mask position with numpy : 2.9325485229492188e-05 nb_pixel_total : 96 time to create 1 rle with old method : 0.0001671314239501953 length of segment : 11 time for calcul the mask position with numpy : 3.24249267578125e-05 nb_pixel_total : 284 time to create 1 rle with old method : 0.000423431396484375 length of segment : 17 time for calcul the mask position with numpy : 3.147125244140625e-05 nb_pixel_total : 179 time to create 1 rle with old method : 0.00028228759765625 length of segment : 15 time for calcul the mask position with numpy : 3.600120544433594e-05 nb_pixel_total : 362 time to create 1 rle with old method : 0.0004954338073730469 length of segment : 26 time for calcul the mask position with numpy : 3.337860107421875e-05 nb_pixel_total : 262 time to create 1 rle with old method : 0.00037026405334472656 length of segment : 27 time for calcul the mask position with numpy : 4.792213439941406e-05 nb_pixel_total : 856 time to create 1 rle with old method : 0.00122833251953125 length of segment : 37 time for calcul the mask position with numpy : 3.170967102050781e-05 nb_pixel_total : 348 time to create 1 rle with old method : 0.0004849433898925781 length of segment : 19 time for calcul the mask position with numpy : 3.62396240234375e-05 nb_pixel_total : 364 time to create 1 rle with old method : 0.0005064010620117188 length of segment : 26 time for calcul the mask position with numpy : 3.075599670410156e-05 nb_pixel_total : 366 time to create 1 rle with old method : 0.0005407333374023438 length of segment : 21 time for calcul the mask position with numpy : 3.814697265625e-05 nb_pixel_total : 740 time to create 1 rle with old method : 0.0010404586791992188 length of segment : 29 time for calcul the mask position with numpy : 3.2901763916015625e-05 nb_pixel_total : 276 time to create 1 rle with old method : 0.0004177093505859375 length of segment : 17 time for calcul the mask position with numpy : 3.170967102050781e-05 nb_pixel_total : 285 time to create 1 rle with old method : 0.0003781318664550781 length of segment : 22 time for calcul the mask position with numpy : 6.67572021484375e-05 nb_pixel_total : 2145 time to create 1 rle with old method : 0.0026483535766601562 length of segment : 52 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.57266 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 14 time for calcul the mask position with numpy : 5.435943603515625e-05 nb_pixel_total : 949 time to create 1 rle with old method : 0.001285552978515625 length of segment : 51 time for calcul the mask position with numpy : 9.012222290039062e-05 nb_pixel_total : 1777 time to create 1 rle with old method : 0.0024001598358154297 length of segment : 73 time for calcul the mask position with numpy : 4.9591064453125e-05 nb_pixel_total : 521 time to create 1 rle with old method : 0.0007426738739013672 length of segment : 28 time for calcul the mask position with numpy : 5.3882598876953125e-05 nb_pixel_total : 233 time to create 1 rle with old method : 0.0005917549133300781 length of segment : 13 time for calcul the mask position with numpy : 5.626678466796875e-05 nb_pixel_total : 385 time to create 1 rle with old method : 0.0007555484771728516 length of segment : 31 time for calcul the mask position with numpy : 3.457069396972656e-05 nb_pixel_total : 145 time to create 1 rle with old method : 0.00023818016052246094 length of segment : 13 time for calcul the mask position with numpy : 3.600120544433594e-05 nb_pixel_total : 252 time to create 1 rle with old method : 0.0004477500915527344 length of segment : 17 time for calcul the mask position with numpy : 4.267692565917969e-05 nb_pixel_total : 541 time to create 1 rle with old method : 0.0008144378662109375 length of segment : 44 time for calcul the mask position with numpy : 4.744529724121094e-05 nb_pixel_total : 957 time to create 1 rle with old method : 0.001295328140258789 length of segment : 35 time for calcul the mask position with numpy : 3.5762786865234375e-05 nb_pixel_total : 158 time to create 1 rle with old method : 0.00026226043701171875 length of segment : 13 time for calcul the mask position with numpy : 3.9577484130859375e-05 nb_pixel_total : 541 time to create 1 rle with old method : 0.000728607177734375 length of segment : 28 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 619 time to create 1 rle with old method : 0.0008034706115722656 length of segment : 43 time for calcul the mask position with numpy : 5.0067901611328125e-05 nb_pixel_total : 1066 time to create 1 rle with old method : 0.0014209747314453125 length of segment : 38 time for calcul the mask position with numpy : 4.482269287109375e-05 nb_pixel_total : 629 time to create 1 rle with old method : 0.0008363723754882812 length of segment : 43 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 140.16250 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 3.4809112548828125e-05 nb_pixel_total : 23 time to create 1 rle with old method : 7.82012939453125e-05 length of segment : 3 time for calcul the mask position with numpy : 0.00012135505676269531 nb_pixel_total : 4 time to create 1 rle with old method : 3.8623809814453125e-05 length of segment : 3 Detection mask done ! Trying to reset tf kernel 717011 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 507 tf kernel not reseted sub process len(results) : 253 len(list_Values) 253 None max_time_sub_proc : 3600 parent process len(results) : 0 len(list_Values) 253 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5774 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Catched exception ! Connect or reconnect ! Number saved : None batch 1 Loaded 253 chid ids of type : 4228 Number RLEs to save : 12280 save missing photos in datou_result : time spend for datou_step_exec : 24.605637788772583 time spend to save output : 1.4759292602539062 total time spend for step 1 : 26.08156704902649 step2:brightness Fri Mar 7 03:20:55 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1741314028_716307_1341329234_e7b053d6572fd84370ef0c1d85d56c57.jpg treat image : temp/1741314028_716307_1341329229_23aa9f984256a72f2b2929f36522781c.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 2 time used for this insertion : 0.013402938842773438 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 2 time used for this insertion : 0.018084287643432617 save missing photos in datou_result : time spend for datou_step_exec : 0.6766517162322998 time spend to save output : 0.036522865295410156 total time spend for step 2 : 0.71317458152771 step3:blur_detection Fri Mar 7 03:20:55 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1741314028_716307_1341329234_e7b053d6572fd84370ef0c1d85d56c57.jpg resize: (1080, 1920) 1341329234 -8.248775659161165 treat image : temp/1741314028_716307_1341329229_23aa9f984256a72f2b2929f36522781c.jpg resize: (1080, 1920) 1341329229 -8.236877990788066 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 2 time used for this insertion : 0.014425277709960938 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 2 time used for this insertion : 0.0199892520904541 save missing photos in datou_result : time spend for datou_step_exec : 2.0062992572784424 time spend to save output : 0.03982830047607422 total time spend for step 3 : 2.0461275577545166 step4:rle_unique_nms_with_priority Fri Mar 7 03:20:57 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 253 chid ids of type : 4228 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 122 nb_hashtags : 8 time to prepare the origin masks : 1.1399338245391846 time for calcul the mask position with numpy : 0.2773263454437256 nb_pixel_total : 1808011 time to create 1 rle with new method : 0.11114954948425293 time for calcul the mask position with numpy : 0.009424924850463867 nb_pixel_total : 47530 time to create 1 rle with old method : 0.05524611473083496 time for calcul the mask position with numpy : 0.008712053298950195 nb_pixel_total : 975 time to create 1 rle with old method : 0.0012006759643554688 time for calcul the mask position with numpy : 0.008518218994140625 nb_pixel_total : 158 time to create 1 rle with old method : 0.0002493858337402344 time for calcul the mask position with numpy : 0.008471012115478516 nb_pixel_total : 1677 time to create 1 rle with old method : 0.002031564712524414 time for calcul the mask position with numpy : 0.008561372756958008 nb_pixel_total : 454 time to create 1 rle with old method : 0.0005695819854736328 time for calcul the mask position with numpy : 0.008586883544921875 nb_pixel_total : 644 time to create 1 rle with old method : 0.0008282661437988281 time for calcul the mask position with numpy : 0.00968480110168457 nb_pixel_total : 125 time to create 1 rle with old method : 0.00019407272338867188 time for calcul the mask position with numpy : 0.008522510528564453 nb_pixel_total : 2457 time to create 1 rle with old method : 0.002992868423461914 time for calcul the mask position with numpy : 0.008594512939453125 nb_pixel_total : 117 time to create 1 rle with old method : 0.0001666545867919922 time for calcul the mask position with numpy : 0.00851750373840332 nb_pixel_total : 663 time to create 1 rle with old method : 0.0008275508880615234 time for calcul the mask position with numpy : 0.008542537689208984 nb_pixel_total : 368 time to create 1 rle with old method : 0.00046896934509277344 time for calcul the mask position with numpy : 0.008541584014892578 nb_pixel_total : 373 time to create 1 rle with old method : 0.000469207763671875 time for calcul the mask position with numpy : 0.008805274963378906 nb_pixel_total : 599 time to create 1 rle with old method : 0.0007977485656738281 time for calcul the mask position with numpy : 0.008799076080322266 nb_pixel_total : 1155 time to create 1 rle with old method : 0.0014405250549316406 time for calcul the mask position with numpy : 0.008898258209228516 nb_pixel_total : 52 time to create 1 rle with old method : 0.00015163421630859375 time for calcul the mask position with numpy : 0.00883030891418457 nb_pixel_total : 383 time to create 1 rle with old method : 0.00048065185546875 time for calcul the mask position with numpy : 0.008815526962280273 nb_pixel_total : 754 time to create 1 rle with old method : 0.0009419918060302734 time for calcul the mask position with numpy : 0.00891566276550293 nb_pixel_total : 1044 time to create 1 rle with old method : 0.0012824535369873047 time for calcul the mask position with numpy : 0.010362625122070312 nb_pixel_total : 4486 time to create 1 rle with old method : 0.0053980350494384766 time for calcul the mask position with numpy : 0.008837223052978516 nb_pixel_total : 422 time to create 1 rle with old method : 0.0006043910980224609 time for calcul the mask position with numpy : 0.008977651596069336 nb_pixel_total : 64 time to create 1 rle with old method : 0.00013947486877441406 time for calcul the mask position with numpy : 0.008547782897949219 nb_pixel_total : 432 time to create 1 rle with old method : 0.0006706714630126953 time for calcul the mask position with numpy : 0.009367942810058594 nb_pixel_total : 799 time to create 1 rle with old method : 0.0009784698486328125 time for calcul the mask position with numpy : 0.008476495742797852 nb_pixel_total : 887 time to create 1 rle with old method : 0.001117706298828125 time for calcul the mask position with numpy : 0.008461236953735352 nb_pixel_total : 1842 time to create 1 rle with old method : 0.0022013187408447266 time for calcul the mask position with numpy : 0.00851893424987793 nb_pixel_total : 83 time to create 1 rle with old method : 0.00012683868408203125 time for calcul the mask position with numpy : 0.00849151611328125 nb_pixel_total : 838 time to create 1 rle with old method : 0.0009982585906982422 time for calcul the mask position with numpy : 0.008455514907836914 nb_pixel_total : 358 time to create 1 rle with old method : 0.0004343986511230469 time for calcul the mask position with numpy : 0.008508920669555664 nb_pixel_total : 541 time to create 1 rle with old method : 0.0006616115570068359 time for calcul the mask position with numpy : 0.008411645889282227 nb_pixel_total : 24 time to create 1 rle with old method : 6.866455078125e-05 time for calcul the mask position with numpy : 0.008434295654296875 nb_pixel_total : 443 time to create 1 rle with old method : 0.0005903244018554688 time for calcul the mask position with numpy : 0.00846242904663086 nb_pixel_total : 151 time to create 1 rle with old method : 0.0001881122589111328 time for calcul the mask position with numpy : 0.008518218994140625 nb_pixel_total : 15406 time to create 1 rle with old method : 0.019347429275512695 time for calcul the mask position with numpy : 0.008673667907714844 nb_pixel_total : 384 time to create 1 rle with old method : 0.0006051063537597656 time for calcul the mask position with numpy : 0.008834123611450195 nb_pixel_total : 725 time to create 1 rle with old method : 0.0009098052978515625 time for calcul the mask position with numpy : 0.008872032165527344 nb_pixel_total : 297 time to create 1 rle with old method : 0.00037860870361328125 time for calcul the mask position with numpy : 0.009027242660522461 nb_pixel_total : 9065 time to create 1 rle with old method : 0.010863542556762695 time for calcul the mask position with numpy : 0.008748531341552734 nb_pixel_total : 1336 time to create 1 rle with old method : 0.0016984939575195312 time for calcul the mask position with numpy : 0.008729934692382812 nb_pixel_total : 41 time to create 1 rle with old method : 8.0108642578125e-05 time for calcul the mask position with numpy : 0.008644580841064453 nb_pixel_total : 619 time to create 1 rle with old method : 0.0007953643798828125 time for calcul the mask position with numpy : 0.008846759796142578 nb_pixel_total : 157 time to create 1 rle with old method : 0.0002262592315673828 time for calcul the mask position with numpy : 0.009207725524902344 nb_pixel_total : 318 time to create 1 rle with old method : 0.0005028247833251953 time for calcul the mask position with numpy : 0.008877038955688477 nb_pixel_total : 217 time to create 1 rle with old method : 0.0002925395965576172 time for calcul the mask position with numpy : 0.01062774658203125 nb_pixel_total : 1304 time to create 1 rle with old method : 0.0017895698547363281 time for calcul the mask position with numpy : 0.009202241897583008 nb_pixel_total : 572 time to create 1 rle with old method : 0.0007390975952148438 time for calcul the mask position with numpy : 0.011540412902832031 nb_pixel_total : 1449 time to create 1 rle with old method : 0.0018148422241210938 time for calcul the mask position with numpy : 0.008840322494506836 nb_pixel_total : 221 time to create 1 rle with old method : 0.00031685829162597656 time for calcul the mask position with numpy : 0.008614540100097656 nb_pixel_total : 3394 time to create 1 rle with old method : 0.004141807556152344 time for calcul the mask position with numpy : 0.008924722671508789 nb_pixel_total : 121 time to create 1 rle with old method : 0.00022745132446289062 time for calcul the mask position with numpy : 0.008853435516357422 nb_pixel_total : 503 time to create 1 rle with old method : 0.0006422996520996094 time for calcul the mask position with numpy : 0.009213447570800781 nb_pixel_total : 674 time to create 1 rle with old method : 0.0010671615600585938 time for calcul the mask position with numpy : 0.009186983108520508 nb_pixel_total : 100 time to create 1 rle with old method : 0.0001666545867919922 time for calcul the mask position with numpy : 0.009243965148925781 nb_pixel_total : 1709 time to create 1 rle with old method : 0.0022644996643066406 time for calcul the mask position with numpy : 0.008938312530517578 nb_pixel_total : 315 time to create 1 rle with old method : 0.0004329681396484375 time for calcul the mask position with numpy : 0.00905919075012207 nb_pixel_total : 375 time to create 1 rle with old method : 0.0006215572357177734 time for calcul the mask position with numpy : 0.00897669792175293 nb_pixel_total : 1186 time to create 1 rle with old method : 0.0017423629760742188 time for calcul the mask position with numpy : 0.009730339050292969 nb_pixel_total : 1276 time to create 1 rle with old method : 0.001760721206665039 time for calcul the mask position with numpy : 0.009432077407836914 nb_pixel_total : 1186 time to create 1 rle with old method : 0.0015516281127929688 time for calcul the mask position with numpy : 0.008983135223388672 nb_pixel_total : 378 time to create 1 rle with old method : 0.0004966259002685547 time for calcul the mask position with numpy : 0.009183168411254883 nb_pixel_total : 3072 time to create 1 rle with old method : 0.003888368606567383 time for calcul the mask position with numpy : 0.00946497917175293 nb_pixel_total : 1519 time to create 1 rle with old method : 0.0018405914306640625 time for calcul the mask position with numpy : 0.008999347686767578 nb_pixel_total : 709 time to create 1 rle with old method : 0.0009341239929199219 time for calcul the mask position with numpy : 0.00865793228149414 nb_pixel_total : 89 time to create 1 rle with old method : 0.0001347064971923828 time for calcul the mask position with numpy : 0.008789300918579102 nb_pixel_total : 291 time to create 1 rle with old method : 0.00038552284240722656 time for calcul the mask position with numpy : 0.010324954986572266 nb_pixel_total : 240 time to create 1 rle with old method : 0.00032329559326171875 time for calcul the mask position with numpy : 0.008636951446533203 nb_pixel_total : 143 time to create 1 rle with old method : 0.0002086162567138672 time for calcul the mask position with numpy : 0.008920907974243164 nb_pixel_total : 27 time to create 1 rle with old method : 6.127357482910156e-05 time for calcul the mask position with numpy : 0.008665323257446289 nb_pixel_total : 97 time to create 1 rle with old method : 0.00014972686767578125 time for calcul the mask position with numpy : 0.009570121765136719 nb_pixel_total : 105003 time to create 1 rle with old method : 0.12281584739685059 time for calcul the mask position with numpy : 0.00872659683227539 nb_pixel_total : 320 time to create 1 rle with old method : 0.00041794776916503906 time for calcul the mask position with numpy : 0.008677959442138672 nb_pixel_total : 144 time to create 1 rle with old method : 0.00020885467529296875 time for calcul the mask position with numpy : 0.00866079330444336 nb_pixel_total : 24 time to create 1 rle with old method : 6.103515625e-05 time for calcul the mask position with numpy : 0.008771181106567383 nb_pixel_total : 248 time to create 1 rle with old method : 0.00031757354736328125 time for calcul the mask position with numpy : 0.008742094039916992 nb_pixel_total : 183 time to create 1 rle with old method : 0.0002601146697998047 time for calcul the mask position with numpy : 0.008997917175292969 nb_pixel_total : 209 time to create 1 rle with old method : 0.0002720355987548828 time for calcul the mask position with numpy : 0.00883626937866211 nb_pixel_total : 217 time to create 1 rle with old method : 0.0002849102020263672 time for calcul the mask position with numpy : 0.008732080459594727 nb_pixel_total : 231 time to create 1 rle with old method : 0.000316619873046875 time for calcul the mask position with numpy : 0.008935928344726562 nb_pixel_total : 782 time to create 1 rle with old method : 0.0009446144104003906 time for calcul the mask position with numpy : 0.008821249008178711 nb_pixel_total : 212 time to create 1 rle with old method : 0.00027871131896972656 time for calcul the mask position with numpy : 0.009533166885375977 nb_pixel_total : 370 time to create 1 rle with old method : 0.0004749298095703125 time for calcul the mask position with numpy : 0.008848428726196289 nb_pixel_total : 175 time to create 1 rle with old method : 0.0002453327178955078 time for calcul the mask position with numpy : 0.008810281753540039 nb_pixel_total : 1248 time to create 1 rle with old method : 0.00154876708984375 time for calcul the mask position with numpy : 0.008798599243164062 nb_pixel_total : 837 time to create 1 rle with old method : 0.0011034011840820312 time for calcul the mask position with numpy : 0.008735418319702148 nb_pixel_total : 382 time to create 1 rle with old method : 0.0005404949188232422 time for calcul the mask position with numpy : 0.008713722229003906 nb_pixel_total : 355 time to create 1 rle with old method : 0.0004634857177734375 time for calcul the mask position with numpy : 0.009361028671264648 nb_pixel_total : 308 time to create 1 rle with old method : 0.0005466938018798828 time for calcul the mask position with numpy : 0.010277748107910156 nb_pixel_total : 605 time to create 1 rle with old method : 0.0011036396026611328 time for calcul the mask position with numpy : 0.010853290557861328 nb_pixel_total : 159 time to create 1 rle with old method : 0.0003249645233154297 time for calcul the mask position with numpy : 0.011293411254882812 nb_pixel_total : 46 time to create 1 rle with old method : 0.00014090538024902344 time for calcul the mask position with numpy : 0.011160135269165039 nb_pixel_total : 4672 time to create 1 rle with old method : 0.00904703140258789 time for calcul the mask position with numpy : 0.011364459991455078 nb_pixel_total : 2027 time to create 1 rle with old method : 0.003731966018676758 time for calcul the mask position with numpy : 0.011965036392211914 nb_pixel_total : 695 time to create 1 rle with old method : 0.001287698745727539 time for calcul the mask position with numpy : 0.01161050796508789 nb_pixel_total : 1563 time to create 1 rle with old method : 0.00287628173828125 time for calcul the mask position with numpy : 0.011599063873291016 nb_pixel_total : 4 time to create 1 rle with old method : 0.0001659393310546875 time for calcul the mask position with numpy : 0.011642932891845703 nb_pixel_total : 23 time to create 1 rle with old method : 0.0001277923583984375 time for calcul the mask position with numpy : 0.011399269104003906 nb_pixel_total : 728 time to create 1 rle with old method : 0.0013952255249023438 time for calcul the mask position with numpy : 0.009982109069824219 nb_pixel_total : 8417 time to create 1 rle with old method : 0.010238885879516602 time for calcul the mask position with numpy : 0.00888681411743164 nb_pixel_total : 214 time to create 1 rle with old method : 0.0002715587615966797 time for calcul the mask position with numpy : 0.008922815322875977 nb_pixel_total : 177 time to create 1 rle with old method : 0.00023984909057617188 time for calcul the mask position with numpy : 0.008866548538208008 nb_pixel_total : 470 time to create 1 rle with old method : 0.0005791187286376953 time for calcul the mask position with numpy : 0.008682966232299805 nb_pixel_total : 62 time to create 1 rle with old method : 9.965896606445312e-05 time for calcul the mask position with numpy : 0.008804082870483398 nb_pixel_total : 212 time to create 1 rle with old method : 0.0002739429473876953 time for calcul the mask position with numpy : 0.008694887161254883 nb_pixel_total : 428 time to create 1 rle with old method : 0.0005555152893066406 time for calcul the mask position with numpy : 0.008650779724121094 nb_pixel_total : 492 time to create 1 rle with old method : 0.0006005764007568359 time for calcul the mask position with numpy : 0.008745193481445312 nb_pixel_total : 4971 time to create 1 rle with old method : 0.0062024593353271484 time for calcul the mask position with numpy : 0.008822917938232422 nb_pixel_total : 1004 time to create 1 rle with old method : 0.0012536048889160156 time for calcul the mask position with numpy : 0.00882863998413086 nb_pixel_total : 935 time to create 1 rle with old method : 0.0012104511260986328 time for calcul the mask position with numpy : 0.008758544921875 nb_pixel_total : 620 time to create 1 rle with old method : 0.0007638931274414062 time for calcul the mask position with numpy : 0.008667469024658203 nb_pixel_total : 394 time to create 1 rle with old method : 0.0004937648773193359 time for calcul the mask position with numpy : 0.008632659912109375 nb_pixel_total : 952 time to create 1 rle with old method : 0.0011584758758544922 time for calcul the mask position with numpy : 0.00877523422241211 nb_pixel_total : 212 time to create 1 rle with old method : 0.00030112266540527344 time for calcul the mask position with numpy : 0.008640289306640625 nb_pixel_total : 990 time to create 1 rle with old method : 0.001302957534790039 time for calcul the mask position with numpy : 0.008682966232299805 nb_pixel_total : 296 time to create 1 rle with old method : 0.0003726482391357422 time for calcul the mask position with numpy : 0.008718490600585938 nb_pixel_total : 169 time to create 1 rle with old method : 0.0002377033233642578 time for calcul the mask position with numpy : 0.008662939071655273 nb_pixel_total : 16 time to create 1 rle with old method : 5.817413330078125e-05 time for calcul the mask position with numpy : 0.008654356002807617 nb_pixel_total : 636 time to create 1 rle with old method : 0.0008246898651123047 time for calcul the mask position with numpy : 0.008683443069458008 nb_pixel_total : 372 time to create 1 rle with old method : 0.00047469139099121094 time for calcul the mask position with numpy : 0.008651494979858398 nb_pixel_total : 411 time to create 1 rle with old method : 0.0005772113800048828 time for calcul the mask position with numpy : 0.008582115173339844 nb_pixel_total : 160 time to create 1 rle with old method : 0.00024318695068359375 time for calcul the mask position with numpy : 0.008985519409179688 nb_pixel_total : 193 time to create 1 rle with old method : 0.00025844573974609375 time for calcul the mask position with numpy : 0.008786678314208984 nb_pixel_total : 530 time to create 1 rle with old method : 0.0006682872772216797 time for calcul the mask position with numpy : 0.008823156356811523 nb_pixel_total : 73 time to create 1 rle with old method : 0.0001385211944580078 create new chi : 1.8424291610717773 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.0073528289794921875 batch 1 Loaded 123 chid ids of type : 4230 Number RLEs to save : 11487 TO DO : save crop sub photo not yet done ! save time : 2.126890182495117 nb_obj : 130 nb_hashtags : 7 time to prepare the origin masks : 2.7088563442230225 time for calcul the mask position with numpy : 0.08410859107971191 nb_pixel_total : 1780711 time to create 1 rle with new method : 0.09421253204345703 time for calcul the mask position with numpy : 0.006845235824584961 nb_pixel_total : 15769 time to create 1 rle with old method : 0.018930912017822266 time for calcul the mask position with numpy : 0.007008790969848633 nb_pixel_total : 49549 time to create 1 rle with old method : 0.05888986587524414 time for calcul the mask position with numpy : 0.007462978363037109 nb_pixel_total : 1146 time to create 1 rle with old method : 0.0013735294342041016 time for calcul the mask position with numpy : 0.007262229919433594 nb_pixel_total : 2040 time to create 1 rle with old method : 0.002474546432495117 time for calcul the mask position with numpy : 0.007097482681274414 nb_pixel_total : 15 time to create 1 rle with old method : 5.888938903808594e-05 time for calcul the mask position with numpy : 0.006998538970947266 nb_pixel_total : 925 time to create 1 rle with old method : 0.001180410385131836 time for calcul the mask position with numpy : 0.0070953369140625 nb_pixel_total : 184 time to create 1 rle with old method : 0.00027251243591308594 time for calcul the mask position with numpy : 0.006760120391845703 nb_pixel_total : 705 time to create 1 rle with old method : 0.0009443759918212891 time for calcul the mask position with numpy : 0.0068547725677490234 nb_pixel_total : 31 time to create 1 rle with old method : 9.274482727050781e-05 time for calcul the mask position with numpy : 0.0070667266845703125 nb_pixel_total : 707 time to create 1 rle with old method : 0.0008585453033447266 time for calcul the mask position with numpy : 0.006745576858520508 nb_pixel_total : 2477 time to create 1 rle with old method : 0.002990245819091797 time for calcul the mask position with numpy : 0.006879329681396484 nb_pixel_total : 13 time to create 1 rle with old method : 5.030632019042969e-05 time for calcul the mask position with numpy : 0.006955146789550781 nb_pixel_total : 754 time to create 1 rle with old method : 0.0009121894836425781 time for calcul the mask position with numpy : 0.006933689117431641 nb_pixel_total : 449 time to create 1 rle with old method : 0.00054931640625 time for calcul the mask position with numpy : 0.006839275360107422 nb_pixel_total : 494 time to create 1 rle with old method : 0.0006289482116699219 time for calcul the mask position with numpy : 0.0077610015869140625 nb_pixel_total : 420 time to create 1 rle with old method : 0.0005099773406982422 time for calcul the mask position with numpy : 0.006525993347167969 nb_pixel_total : 724 time to create 1 rle with old method : 0.0008816719055175781 time for calcul the mask position with numpy : 0.006689786911010742 nb_pixel_total : 1077 time to create 1 rle with old method : 0.001329183578491211 time for calcul the mask position with numpy : 0.006745100021362305 nb_pixel_total : 5279 time to create 1 rle with old method : 0.006712913513183594 time for calcul the mask position with numpy : 0.006852149963378906 nb_pixel_total : 404 time to create 1 rle with old method : 0.0004951953887939453 time for calcul the mask position with numpy : 0.00654292106628418 nb_pixel_total : 67 time to create 1 rle with old method : 0.00014710426330566406 time for calcul the mask position with numpy : 0.006598711013793945 nb_pixel_total : 485 time to create 1 rle with old method : 0.0006206035614013672 time for calcul the mask position with numpy : 0.007854223251342773 nb_pixel_total : 1055 time to create 1 rle with old method : 0.0017876625061035156 time for calcul the mask position with numpy : 0.006810188293457031 nb_pixel_total : 4007 time to create 1 rle with old method : 0.004810810089111328 time for calcul the mask position with numpy : 0.006775617599487305 nb_pixel_total : 445 time to create 1 rle with old method : 0.0005662441253662109 time for calcul the mask position with numpy : 0.006842374801635742 nb_pixel_total : 2109 time to create 1 rle with old method : 0.002522706985473633 time for calcul the mask position with numpy : 0.0068149566650390625 nb_pixel_total : 6286 time to create 1 rle with old method : 0.007515907287597656 time for calcul the mask position with numpy : 0.0067369937896728516 nb_pixel_total : 753 time to create 1 rle with old method : 0.0009365081787109375 time for calcul the mask position with numpy : 0.006353616714477539 nb_pixel_total : 44 time to create 1 rle with old method : 0.00010561943054199219 time for calcul the mask position with numpy : 0.0077474117279052734 nb_pixel_total : 510 time to create 1 rle with old method : 0.0006971359252929688 time for calcul the mask position with numpy : 0.010732173919677734 nb_pixel_total : 407 time to create 1 rle with old method : 0.0005381107330322266 time for calcul the mask position with numpy : 0.01068258285522461 nb_pixel_total : 121 time to create 1 rle with old method : 0.0001971721649169922 time for calcul the mask position with numpy : 0.01675271987915039 nb_pixel_total : 147 time to create 1 rle with old method : 0.0002598762512207031 time for calcul the mask position with numpy : 0.01296377182006836 nb_pixel_total : 290 time to create 1 rle with old method : 0.000423431396484375 time for calcul the mask position with numpy : 0.012819290161132812 nb_pixel_total : 9500 time to create 1 rle with old method : 0.011582374572753906 time for calcul the mask position with numpy : 0.015527963638305664 nb_pixel_total : 640 time to create 1 rle with old method : 0.0008294582366943359 time for calcul the mask position with numpy : 0.011819839477539062 nb_pixel_total : 749 time to create 1 rle with old method : 0.0009598731994628906 time for calcul the mask position with numpy : 0.01281118392944336 nb_pixel_total : 288 time to create 1 rle with old method : 0.00041294097900390625 time for calcul the mask position with numpy : 0.012343168258666992 nb_pixel_total : 198 time to create 1 rle with old method : 0.00028896331787109375 time for calcul the mask position with numpy : 0.0076198577880859375 nb_pixel_total : 1311 time to create 1 rle with old method : 0.0015778541564941406 time for calcul the mask position with numpy : 0.007391691207885742 nb_pixel_total : 517 time to create 1 rle with old method : 0.0006353855133056641 time for calcul the mask position with numpy : 0.0070650577545166016 nb_pixel_total : 8075 time to create 1 rle with old method : 0.009783029556274414 time for calcul the mask position with numpy : 0.0077953338623046875 nb_pixel_total : 236 time to create 1 rle with old method : 0.00045418739318847656 time for calcul the mask position with numpy : 0.007955312728881836 nb_pixel_total : 51 time to create 1 rle with old method : 0.00017833709716796875 time for calcul the mask position with numpy : 0.007638454437255859 nb_pixel_total : 1048 time to create 1 rle with old method : 0.0019779205322265625 time for calcul the mask position with numpy : 0.007503986358642578 nb_pixel_total : 447 time to create 1 rle with old method : 0.0009245872497558594 time for calcul the mask position with numpy : 0.007722616195678711 nb_pixel_total : 1008 time to create 1 rle with old method : 0.0021762847900390625 time for calcul the mask position with numpy : 0.007865667343139648 nb_pixel_total : 4055 time to create 1 rle with old method : 0.0074007511138916016 time for calcul the mask position with numpy : 0.0076446533203125 nb_pixel_total : 533 time to create 1 rle with old method : 0.0010335445404052734 time for calcul the mask position with numpy : 0.015990734100341797 nb_pixel_total : 924 time to create 1 rle with old method : 0.0015234947204589844 time for calcul the mask position with numpy : 0.009533882141113281 nb_pixel_total : 321 time to create 1 rle with old method : 0.0007202625274658203 time for calcul the mask position with numpy : 0.00718379020690918 nb_pixel_total : 117 time to create 1 rle with old method : 0.00021123886108398438 time for calcul the mask position with numpy : 0.006815671920776367 nb_pixel_total : 2681 time to create 1 rle with old method : 0.0032749176025390625 time for calcul the mask position with numpy : 0.006750345230102539 nb_pixel_total : 327 time to create 1 rle with old method : 0.0004794597625732422 time for calcul the mask position with numpy : 0.00705718994140625 nb_pixel_total : 1206 time to create 1 rle with old method : 0.0015187263488769531 time for calcul the mask position with numpy : 0.006493568420410156 nb_pixel_total : 17 time to create 1 rle with old method : 4.267692565917969e-05 time for calcul the mask position with numpy : 0.006597042083740234 nb_pixel_total : 17194 time to create 1 rle with old method : 0.01987743377685547 time for calcul the mask position with numpy : 0.00664210319519043 nb_pixel_total : 1509 time to create 1 rle with old method : 0.001842498779296875 time for calcul the mask position with numpy : 0.00781559944152832 nb_pixel_total : 708 time to create 1 rle with old method : 0.0009033679962158203 time for calcul the mask position with numpy : 0.010821342468261719 nb_pixel_total : 36 time to create 1 rle with old method : 0.00011229515075683594 time for calcul the mask position with numpy : 0.007568836212158203 nb_pixel_total : 220 time to create 1 rle with old method : 0.00044918060302734375 time for calcul the mask position with numpy : 0.0074236392974853516 nb_pixel_total : 23 time to create 1 rle with old method : 8.273124694824219e-05 time for calcul the mask position with numpy : 0.007332324981689453 nb_pixel_total : 75 time to create 1 rle with old method : 0.00017571449279785156 time for calcul the mask position with numpy : 0.0073854923248291016 nb_pixel_total : 285 time to create 1 rle with old method : 0.0005412101745605469 time for calcul the mask position with numpy : 0.0074193477630615234 nb_pixel_total : 268 time to create 1 rle with old method : 0.0005288124084472656 time for calcul the mask position with numpy : 0.009903907775878906 nb_pixel_total : 336 time to create 1 rle with old method : 0.0006425380706787109 time for calcul the mask position with numpy : 0.0071299076080322266 nb_pixel_total : 109 time to create 1 rle with old method : 0.0002281665802001953 time for calcul the mask position with numpy : 0.0073735713958740234 nb_pixel_total : 29 time to create 1 rle with old method : 9.131431579589844e-05 time for calcul the mask position with numpy : 0.008962392807006836 nb_pixel_total : 105202 time to create 1 rle with old method : 0.12805700302124023 time for calcul the mask position with numpy : 0.00813913345336914 nb_pixel_total : 89 time to create 1 rle with old method : 0.00012254714965820312 time for calcul the mask position with numpy : 0.0062465667724609375 nb_pixel_total : 233 time to create 1 rle with old method : 0.00032782554626464844 time for calcul the mask position with numpy : 0.006222248077392578 nb_pixel_total : 14 time to create 1 rle with old method : 5.078315734863281e-05 time for calcul the mask position with numpy : 0.006502628326416016 nb_pixel_total : 145 time to create 1 rle with old method : 0.00022125244140625 time for calcul the mask position with numpy : 0.0068225860595703125 nb_pixel_total : 23 time to create 1 rle with old method : 6.079673767089844e-05 time for calcul the mask position with numpy : 0.0076024532318115234 nb_pixel_total : 188 time to create 1 rle with old method : 0.00024628639221191406 time for calcul the mask position with numpy : 0.0064814090728759766 nb_pixel_total : 96 time to create 1 rle with old method : 0.0001354217529296875 time for calcul the mask position with numpy : 0.006577014923095703 nb_pixel_total : 201 time to create 1 rle with old method : 0.00030803680419921875 time for calcul the mask position with numpy : 0.0074748992919921875 nb_pixel_total : 423 time to create 1 rle with old method : 0.0007236003875732422 time for calcul the mask position with numpy : 0.007314205169677734 nb_pixel_total : 1879 time to create 1 rle with old method : 0.0028853416442871094 time for calcul the mask position with numpy : 0.0076901912689208984 nb_pixel_total : 145 time to create 1 rle with old method : 0.0002582073211669922 time for calcul the mask position with numpy : 0.007530927658081055 nb_pixel_total : 113 time to create 1 rle with old method : 0.00028967857360839844 time for calcul the mask position with numpy : 0.0071086883544921875 nb_pixel_total : 252 time to create 1 rle with old method : 0.0005083084106445312 time for calcul the mask position with numpy : 0.007900238037109375 nb_pixel_total : 477 time to create 1 rle with old method : 0.0006322860717773438 time for calcul the mask position with numpy : 0.006731748580932617 nb_pixel_total : 453 time to create 1 rle with old method : 0.0005609989166259766 time for calcul the mask position with numpy : 0.0067555904388427734 nb_pixel_total : 4 time to create 1 rle with old method : 2.6702880859375e-05 time for calcul the mask position with numpy : 0.006609916687011719 nb_pixel_total : 1276 time to create 1 rle with old method : 0.0016896724700927734 time for calcul the mask position with numpy : 0.0067942142486572266 nb_pixel_total : 166 time to create 1 rle with old method : 0.0003349781036376953 time for calcul the mask position with numpy : 0.006835460662841797 nb_pixel_total : 740 time to create 1 rle with old method : 0.0014455318450927734 time for calcul the mask position with numpy : 0.007619142532348633 nb_pixel_total : 368 time to create 1 rle with old method : 0.0004811286926269531 time for calcul the mask position with numpy : 0.007053375244140625 nb_pixel_total : 362 time to create 1 rle with old method : 0.0004906654357910156 time for calcul the mask position with numpy : 0.012868881225585938 nb_pixel_total : 305 time to create 1 rle with old method : 0.0005791187286376953 time for calcul the mask position with numpy : 0.012341022491455078 nb_pixel_total : 1 time to create 1 rle with old method : 4.00543212890625e-05 time for calcul the mask position with numpy : 0.011577844619750977 nb_pixel_total : 179 time to create 1 rle with old method : 0.00024700164794921875 time for calcul the mask position with numpy : 0.011345624923706055 nb_pixel_total : 325 time to create 1 rle with old method : 0.00044274330139160156 time for calcul the mask position with numpy : 0.011236429214477539 nb_pixel_total : 8 time to create 1 rle with old method : 4.601478576660156e-05 time for calcul the mask position with numpy : 0.010021448135375977 nb_pixel_total : 3141 time to create 1 rle with old method : 0.003713846206665039 time for calcul the mask position with numpy : 0.006993293762207031 nb_pixel_total : 2145 time to create 1 rle with old method : 0.002654552459716797 time for calcul the mask position with numpy : 0.006902933120727539 nb_pixel_total : 704 time to create 1 rle with old method : 0.0008916854858398438 time for calcul the mask position with numpy : 0.0068492889404296875 nb_pixel_total : 191 time to create 1 rle with old method : 0.0002543926239013672 time for calcul the mask position with numpy : 0.0077724456787109375 nb_pixel_total : 1624 time to create 1 rle with old method : 0.0019702911376953125 time for calcul the mask position with numpy : 0.008342504501342773 nb_pixel_total : 774 time to create 1 rle with old method : 0.0009522438049316406 time for calcul the mask position with numpy : 0.007761955261230469 nb_pixel_total : 266 time to create 1 rle with old method : 0.00033783912658691406 time for calcul the mask position with numpy : 0.007071495056152344 nb_pixel_total : 178 time to create 1 rle with old method : 0.000293731689453125 time for calcul the mask position with numpy : 0.00746464729309082 nb_pixel_total : 194 time to create 1 rle with old method : 0.000362396240234375 time for calcul the mask position with numpy : 0.009148359298706055 nb_pixel_total : 385 time to create 1 rle with old method : 0.0009250640869140625 time for calcul the mask position with numpy : 0.01060628890991211 nb_pixel_total : 1 time to create 1 rle with old method : 3.504753112792969e-05 time for calcul the mask position with numpy : 0.009343624114990234 nb_pixel_total : 53 time to create 1 rle with old method : 0.00010514259338378906 time for calcul the mask position with numpy : 0.008264541625976562 nb_pixel_total : 221 time to create 1 rle with old method : 0.00032830238342285156 time for calcul the mask position with numpy : 0.007515907287597656 nb_pixel_total : 400 time to create 1 rle with old method : 0.0005881786346435547 time for calcul the mask position with numpy : 0.007374286651611328 nb_pixel_total : 20 time to create 1 rle with old method : 6.175041198730469e-05 time for calcul the mask position with numpy : 0.00824737548828125 nb_pixel_total : 521 time to create 1 rle with old method : 0.0009019374847412109 time for calcul the mask position with numpy : 0.008799552917480469 nb_pixel_total : 5070 time to create 1 rle with old method : 0.008714675903320312 time for calcul the mask position with numpy : 0.007302284240722656 nb_pixel_total : 1777 time to create 1 rle with old method : 0.002458333969116211 time for calcul the mask position with numpy : 0.007536888122558594 nb_pixel_total : 574 time to create 1 rle with old method : 0.0009243488311767578 time for calcul the mask position with numpy : 0.007439136505126953 nb_pixel_total : 541 time to create 1 rle with old method : 0.0007383823394775391 time for calcul the mask position with numpy : 0.007806301116943359 nb_pixel_total : 111 time to create 1 rle with old method : 0.00022101402282714844 time for calcul the mask position with numpy : 0.007843732833862305 nb_pixel_total : 951 time to create 1 rle with old method : 0.0013518333435058594 time for calcul the mask position with numpy : 0.007431507110595703 nb_pixel_total : 262 time to create 1 rle with old method : 0.0003609657287597656 time for calcul the mask position with numpy : 0.007844924926757812 nb_pixel_total : 949 time to create 1 rle with old method : 0.0013816356658935547 time for calcul the mask position with numpy : 0.007321596145629883 nb_pixel_total : 221 time to create 1 rle with old method : 0.0002951622009277344 time for calcul the mask position with numpy : 0.0070760250091552734 nb_pixel_total : 284 time to create 1 rle with old method : 0.0004241466522216797 time for calcul the mask position with numpy : 0.007325172424316406 nb_pixel_total : 3 time to create 1 rle with old method : 3.3855438232421875e-05 time for calcul the mask position with numpy : 0.007511138916015625 nb_pixel_total : 33 time to create 1 rle with old method : 6.771087646484375e-05 time for calcul the mask position with numpy : 0.009648799896240234 nb_pixel_total : 31 time to create 1 rle with old method : 9.417533874511719e-05 time for calcul the mask position with numpy : 0.009619712829589844 nb_pixel_total : 619 time to create 1 rle with old method : 0.0008747577667236328 time for calcul the mask position with numpy : 0.009105205535888672 nb_pixel_total : 348 time to create 1 rle with old method : 0.0004608631134033203 time for calcul the mask position with numpy : 0.00941324234008789 nb_pixel_total : 181 time to create 1 rle with old method : 0.0002760887145996094 time for calcul the mask position with numpy : 0.009293317794799805 nb_pixel_total : 856 time to create 1 rle with old method : 0.0012786388397216797 time for calcul the mask position with numpy : 0.009038448333740234 nb_pixel_total : 570 time to create 1 rle with old method : 0.0008037090301513672 time for calcul the mask position with numpy : 0.009269237518310547 nb_pixel_total : 94 time to create 1 rle with old method : 0.0001633167266845703 create new chi : 1.620563268661499 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.002569437026977539 batch 1 Loaded 131 chid ids of type : 4230 Number RLEs to save : 11978 TO DO : save crop sub photo not yet done ! save time : 1.1413280963897705 map_output_result : {1341329234: (0.0, 'Should be the crop_list due to order', 0.0), 1341329229: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1341329234, 1341329229] Looping around the photos to save general results len do output : 2 /1341329234.Didn't retrieve data . /1341329229.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2646451') ('4234', '21130892', '1341329234', None, None, None, None, None, '2646451') ('4234', None, None, None, None, None, None, None, '2646451') ('4234', '21130892', '1341329229', None, None, None, None, None, '2646451') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.015493631362915039 save_final save missing photos in datou_result : time spend for datou_step_exec : 11.123472929000854 time spend to save output : 0.015980958938598633 total time spend for step 4 : 11.139453887939453 step5:crop_condition Fri Mar 7 03:21:09 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4230 Loading chi in step crop for list_pids : 2 ! batch 1 Loaded 254 chid ids of type : 4230 begin to crop the class : papier param for this class : {'min_score': 0.6} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 90 About to insert : list_path_to_insert length 90 new photo from crops ! About to upload 90 photos upload in portfolio : 4869462 init cache_photo without model_param we have 90 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741314070_716307 we have uploaded 90 photos in the portfolio 4869462 time of upload the photos Elapsed time : 29.44977331161499 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.6} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 32 About to insert : list_path_to_insert length 32 new photo from crops ! About to upload 32 photos upload in portfolio : 4869462 init cache_photo without model_param we have 32 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741314100_716307 we have uploaded 32 photos in the portfolio 4869462 time of upload the photos Elapsed time : 8.432379007339478 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.6} filtre for class : metal hashtag_id of this class : 492628673 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741314109_716307 we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.5686306953430176 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.6} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! About to upload 9 photos upload in portfolio : 4869462 init cache_photo without model_param we have 9 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741314110_716307 we have uploaded 9 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.8624978065490723 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.6} filtre for class : autre hashtag_id of this class : 494826614 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 34 About to insert : list_path_to_insert length 34 new photo from crops ! About to upload 34 photos upload in portfolio : 4869462 init cache_photo without model_param we have 34 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741314113_716307 we have uploaded 34 photos in the portfolio 4869462 time of upload the photos Elapsed time : 15.055394887924194 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.6} filtre for class : pehd hashtag_id of this class : 628944319 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741314128_716307 we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.5364727973937988 we have finished the crop for the class : pehd begin to crop the class : pet_fonce param for this class : {'min_score': 0.6} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1341329234, 1341329229] Looping around the photos to save general results len do output : 167 /1341372182Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372183Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372184Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372185Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372186Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372187Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372188Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372189Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372190Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372191Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372192Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372193Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372194Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372195Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372196Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372198Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372199Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372200Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372201Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372202Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372203Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372204Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372205Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372206Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372207Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372208Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372209Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372210Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372211Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372212Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372213Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372214Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372215Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372216Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372217Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372218Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372219Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372220Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372222Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372223Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372224Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372225Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372226Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372227Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372228Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372229Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372230Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372231Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372232Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372233Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372234Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372235Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372236Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372237Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372238Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372239Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372240Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372241Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372242Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372244Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372245Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372246Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372247Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372248Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372249Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372250Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372251Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372252Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372253Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372254Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372255Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372256Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372257Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372258Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372259Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372260Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372262Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372263Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372264Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372265Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372266Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372267Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372268Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372269Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372270Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372271Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372272Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372273Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372274Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372275Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372280Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372281Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372282Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372283Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372284Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372285Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372286Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372288Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372291Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372292Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372293Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372294Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372295Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372296Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372297Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372298Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372299Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372300Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372301Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372302Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372303Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372304Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372305Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372306Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372307Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372308Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372309Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372311Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372312Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372313Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372314Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372315Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372316Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372322Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372323Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372324Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372325Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372326Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372327Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372328Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372329Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372330Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372364Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372365Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372366Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372367Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372368Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372369Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372370Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372371Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372372Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372373Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372374Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372375Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372376Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372377Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372378Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372379Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372380Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372381Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372382Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372383Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372384Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372385Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372386Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372387Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372388Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372389Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372390Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372391Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372392Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372393Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372394Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372395Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372396Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372397Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341372399Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2646451') ('4234', '21130892', '1341329234', None, None, None, None, None, '2646451') ('4234', None, None, None, None, None, None, None, '2646451') ('4234', '21130892', '1341329229', None, None, None, None, None, '2646451') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 503 time used for this insertion : 0.7189099788665771 save_final save missing photos in datou_result : time spend for datou_step_exec : 60.11947536468506 time spend to save output : 0.7244682312011719 total time spend for step 5 : 60.84394359588623 step6:thcl Fri Mar 7 03:22:09 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! we are using the classfication for only one thcl 3237 time to import caffe and check if the image exist : 0.008411645889282227 time to convert the images to numpy array : 0.03081512451171875 time to import caffe and check if the image exist : 0.009382009506225586 time to convert the images to numpy array : 0.03287220001220703 time to import caffe and check if the image exist : 0.010436773300170898 time to convert the images to numpy array : 0.03432774543762207 time to import caffe and check if the image exist : 0.014283180236816406 time to convert the images to numpy array : 0.03233003616333008 time to import caffe and check if the image exist : 0.013998985290527344 time to convert the images to numpy array : 0.03677797317504883 time to import caffe and check if the image exist : 0.00979757308959961 time to convert the images to numpy array : 0.0415492057800293 time to import caffe and check if the image exist : 0.018341779708862305 time to convert the images to numpy array : 0.033339500427246094 time to import caffe and check if the image exist : 0.011240243911743164 time to convert the images to numpy array : 0.040946245193481445 time to import caffe and check if the image exist : 0.01531982421875 time to convert the images to numpy array : 0.03877735137939453 time to import caffe and check if the image exist : 0.022916555404663086 time to convert the images to numpy array : 0.03206658363342285 total time to convert the images to numpy array : 0.27980589866638184 list photo_ids error: [] list photo_ids correct : [1341372235, 1341372236, 1341372237, 1341372238, 1341372239, 1341372240, 1341372241, 1341372242, 1341372244, 1341372245, 1341372246, 1341372247, 1341372248, 1341372249, 1341372250, 1341372251, 1341372252, 1341372271, 1341372272, 1341372273, 1341372274, 1341372275, 1341372280, 1341372281, 1341372282, 1341372283, 1341372284, 1341372285, 1341372286, 1341372288, 1341372291, 1341372292, 1341372293, 1341372294, 1341372200, 1341372201, 1341372202, 1341372203, 1341372204, 1341372205, 1341372206, 1341372207, 1341372208, 1341372209, 1341372210, 1341372211, 1341372212, 1341372213, 1341372214, 1341372215, 1341372216, 1341372385, 1341372386, 1341372387, 1341372388, 1341372389, 1341372390, 1341372391, 1341372392, 1341372393, 1341372394, 1341372395, 1341372396, 1341372397, 1341372399, 1341372217, 1341372218, 1341372219, 1341372220, 1341372222, 1341372223, 1341372224, 1341372225, 1341372226, 1341372227, 1341372228, 1341372229, 1341372230, 1341372231, 1341372232, 1341372233, 1341372234, 1341372182, 1341372183, 1341372184, 1341372185, 1341372186, 1341372187, 1341372188, 1341372189, 1341372190, 1341372191, 1341372192, 1341372193, 1341372194, 1341372195, 1341372196, 1341372198, 1341372199, 1341372368, 1341372369, 1341372370, 1341372371, 1341372372, 1341372373, 1341372374, 1341372375, 1341372376, 1341372377, 1341372378, 1341372379, 1341372380, 1341372381, 1341372382, 1341372383, 1341372384, 1341372313, 1341372314, 1341372315, 1341372316, 1341372322, 1341372323, 1341372324, 1341372325, 1341372326, 1341372327, 1341372328, 1341372329, 1341372330, 1341372364, 1341372365, 1341372366, 1341372367, 1341372253, 1341372254, 1341372255, 1341372256, 1341372257, 1341372258, 1341372259, 1341372260, 1341372262, 1341372263, 1341372264, 1341372265, 1341372266, 1341372267, 1341372268, 1341372269, 1341372270, 1341372295, 1341372296, 1341372297, 1341372298, 1341372299, 1341372300, 1341372301, 1341372302, 1341372303, 1341372304, 1341372305, 1341372306, 1341372307, 1341372308, 1341372309, 1341372311, 1341372312] number of photos to traite : 167 try to delete the photos incorrect in DB tagging for thcl : 3237 To do loadFromThcl(), then load ParamDescType : thcl3237 thcls : [{'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'}] thcl {'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'} Update svm_hashtag_type_desc : 5561 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) To loadFromThcl() : net_5561 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10151 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_rubbia_refus_2500 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_rubbia_refus_2500 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_rubbia_refus_2500 /data/models_weight/learn_rubbia_refus_2500/caffemodel size_local : 94358479 size in s3 : 94358479 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:16 caffemodel already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt size_local : 32544 size in s3 : 32544 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:15 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:09:52 mean.npy already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/synset_words.txt size_local : 334 size in s3 : 334 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:10:06 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt caffemodel_filename : /data/models_weight/learn_rubbia_refus_2500/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 9932 max_wait_temp : 1 max_wait : 0 dict_keys(['pool5', 'prob']) time used to do the prepocess of the images : 0.81697678565979 time used to do the prediction : 0.8166677951812744 save descriptor for thcl : 3237 time to traite the descriptors : 0.9816536903381348 Catched exception ! Connect or reconnect ! storage_type for insertDescriptorsMulti : 3 To insert : 1341372235 To insert : 1341372236 To insert : 1341372237 To insert : 1341372238 To insert : 1341372239 To insert : 1341372240 To insert : 1341372241 To insert : 1341372242 To insert : 1341372244 To insert : 1341372245 To insert : 1341372246 To insert : 1341372247 To insert : 1341372248 To insert : 1341372249 To insert : 1341372250 To insert : 1341372251 To insert : 1341372252 To insert : 1341372271 To insert : 1341372272 To insert : 1341372273 To insert : 1341372274 To insert : 1341372275 To insert : 1341372280 To insert : 1341372281 To insert : 1341372282 To insert : 1341372283 To insert : 1341372284 To insert : 1341372285 To insert : 1341372286 To insert : 1341372288 To insert : 1341372291 To insert : 1341372292 To insert : 1341372293 To insert : 1341372294 To insert : 1341372200 To insert : 1341372201 To insert : 1341372202 To insert : 1341372203 To insert : 1341372204 To insert : 1341372205 To insert : 1341372206 To insert : 1341372207 To insert : 1341372208 To insert : 1341372209 To insert : 1341372210 To insert : 1341372211 To insert : 1341372212 To insert : 1341372213 To insert : 1341372214 To insert : 1341372215 To insert : 1341372216 To insert : 1341372385 To insert : 1341372386 To insert : 1341372387 To insert : 1341372388 To insert : 1341372389 To insert : 1341372390 To insert : 1341372391 To insert : 1341372392 To insert : 1341372393 To insert : 1341372394 To insert : 1341372395 To insert : 1341372396 To insert : 1341372397 To insert : 1341372399 To insert : 1341372217 To insert : 1341372218 To insert : 1341372219 To insert : 1341372220 To insert : 1341372222 To insert : 1341372223 To insert : 1341372224 To insert : 1341372225 To insert : 1341372226 To insert : 1341372227 To insert : 1341372228 To insert : 1341372229 To insert : 1341372230 To insert : 1341372231 To insert : 1341372232 To insert : 1341372233 To insert : 1341372234 To insert : 1341372182 To insert : 1341372183 To insert : 1341372184 To insert : 1341372185 To insert : 1341372186 To insert : 1341372187 To insert : 1341372188 To insert : 1341372189 To insert : 1341372190 To insert : 1341372191 To insert : 1341372192 To insert : 1341372193 To insert : 1341372194 To insert : 1341372195 To insert : 1341372196 To insert : 1341372198 To insert : 1341372199 To insert : 1341372368 To insert : 1341372369 To insert : 1341372370 To insert : 1341372371 To insert : 1341372372 To insert : 1341372373 To insert : 1341372374 To insert : 1341372375 To insert : 1341372376 To insert : 1341372377 To insert : 1341372378 To insert : 1341372379 To insert : 1341372380 To insert : 1341372381 To insert : 1341372382 To insert : 1341372383 To insert : 1341372384 To insert : 1341372313 To insert : 1341372314 To insert : 1341372315 To insert : 1341372316 To insert : 1341372322 To insert : 1341372323 To insert : 1341372324 To insert : 1341372325 To insert : 1341372326 To insert : 1341372327 To insert : 1341372328 To insert : 1341372329 To insert : 1341372330 To insert : 1341372364 To insert : 1341372365 To insert : 1341372366 To insert : 1341372367 To insert : 1341372253 To insert : 1341372254 To insert : 1341372255 To insert : 1341372256 To insert : 1341372257 To insert : 1341372258 To insert : 1341372259 To insert : 1341372260 To insert : 1341372262 To insert : 1341372263 To insert : 1341372264 To insert : 1341372265 To insert : 1341372266 To insert : 1341372267 To insert : 1341372268 To insert : 1341372269 To insert : 1341372270 To insert : 1341372295 To insert : 1341372296 To insert : 1341372297 To insert : 1341372298 To insert : 1341372299 To insert : 1341372300 To insert : 1341372301 To insert : 1341372302 To insert : 1341372303 To insert : 1341372304 To insert : 1341372305 To insert : 1341372306 To insert : 1341372307 To insert : 1341372308 To insert : 1341372309 To insert : 1341372311 To insert : 1341372312 time to insert the descriptors : 55.09242558479309 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos SAVE THCL : begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 239 time used for this insertion : 0.023645639419555664 save missing photos in datou_result : time spend for datou_step_exec : 61.98654842376709 time spend to save output : 0.07798194885253906 total time spend for step 6 : 62.06453037261963 step7:ventilate_hashtags_in_portfolio Fri Mar 7 03:23:11 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 21130892 get user id for portfolio 21130892 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21130892 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Tetrapak','pehd','flou','carton','Carton','Film_plastique','environnement','autre','Papier','PET_fonce','mal_croppe','PET_clair','papier','pet_clair','pet_fonce','refus','metal','PEHD')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21130892 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Tetrapak','pehd','flou','carton','Carton','Film_plastique','environnement','autre','Papier','PET_fonce','mal_croppe','PET_clair','papier','pet_clair','pet_fonce','refus','metal','PEHD')) AND mptpi.`min_score`=0.6 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21130892 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Tetrapak','pehd','flou','carton','Carton','Film_plastique','environnement','autre','Papier','PET_fonce','mal_croppe','PET_clair','papier','pet_clair','pet_fonce','refus','metal','PEHD')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21130892 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Tetrapak','pehd','flou','carton','Carton','Film_plastique','environnement','autre','Papier','PET_fonce','mal_croppe','PET_clair','papier','pet_clair','pet_fonce','refus','metal','PEHD')) AND mptpi.`min_score`=0.6 To do lien utilise dans velours : https://www.fotonower.com/velours/21133571,21133588,21133573,21133575,21133576,21133577,21133578,21133583,21133585,21133581,21133584,21133586,21133587?tags=tetrapak,pehd,flou,carton,film_plastique,environnement,autre,papier,pet_fonce,mal_croppe,pet_clair,refus,metal&datou_id_consolidate=4235&port_consolidate=21130892 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1341329234, 1341329229] Looping around the photos to save general results len do output : 1 /21130892. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2646451') ('4234', '21130892', '1341329234', None, None, None, None, None, '2646451') ('4234', None, None, None, None, None, None, None, '2646451') ('4234', '21130892', '1341329229', None, None, None, None, None, '2646451') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.014314889907836914 save_final save missing photos in datou_result : time spend for datou_step_exec : 5.122217416763306 time spend to save output : 0.014906167984008789 total time spend for step 7 : 5.1371235847473145 step8:final Fri Mar 7 03:23:17 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1341329234: ('0.22029002069900358',), 1341329229: ('0.22029002069900358',)} new output for save of step final : {1341329234: ('0.22029002069900358',), 1341329229: ('0.22029002069900358',)} [1341329234, 1341329229] Looping around the photos to save general results len do output : 2 /1341329234.Didn't retrieve data . /1341329229.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2646451') ('4234', '21130892', '1341329234', None, None, None, None, None, '2646451') ('4234', None, None, None, None, None, None, None, '2646451') ('4234', '21130892', '1341329229', None, None, None, None, None, '2646451') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.015107870101928711 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.11959218978881836 time spend to save output : 0.015341043472290039 total time spend for step 8 : 0.1349332332611084 step9:velours_tree Fri Mar 7 03:23:17 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure list_portfolios : 21133571,21133588,21133573,21133575,21133576,21133577,21133578,21133583,21133585,21133581,21133584,21133586,21133587 photo desc type : 5561 - Retrieving photos to tag... query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21133571 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21133588 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21133573 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21133575 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21133576 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21133577 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21133578 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21133583 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21133585 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21133581 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21133584 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21133586 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21133587 ORDER BY ph.size desc - Loading descriptors... Size : 2048 len(descriptors) : 167 Compute structured hierarchical clustering... ward : AgglomerativeClustering(n_clusters=6) ward.labels_ : [0 4 0 0 4 0 0 0 2 4 0 5 2 2 2 4 2 2 4 2 4 2 2 4 1 2 2 5 2 2 2 4 4 1 1 2 1 5 2 2 4 5 0 5 3 0 0 0 4 0 0 4 2 1 4 0 0 0 4 2 2 4 2 2 4 4 2 4 2 4 1 2 2 5 2 2 2 4 4 1 1 2 5 2 4 2 4 5 0 2 0 0 2 2 1 5 1 1 5 4 2 3 0 4 0 2 2 2 1 5 0 1 2 2 4 5 4 2 2 3 4 4 4 0 4 4 5 4 4 4 4 0 0 3 3 0 2 3 1 4 1 0 2 0 2 1 5 5 0 0 0 0 0 3 3 0 3 1 4 5 1 0 2 0 0 0 2] Elapsed time: 0.025812149047851562 graph_id used : 81841 - Beta version, working pretty good on 11-5-16 ! https://www.fotonower.com/velours/21133571,21133588,21133573,21133575,21133576,21133577,21133578,21133583,21133585,21133581,21133584,21133586,21133587?tags=tetrapak,pehd,flou,carton,film_plastique,environnement,autre,papier,pet_fonce,mal_croppe,pet_clair,refus,metal&datou_id_consolidate=4235&port_consolidate=21130892&tree_id=81841 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : velours_tree we use saveGeneral [1341329234, 1341329229] Looping around the photos to save general results len do output : 1 /21130892Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2646451') ('4234', '21130892', '1341329234', None, None, None, None, None, '2646451') ('4234', None, None, None, None, None, None, None, '2646451') ('4234', '21130892', '1341329229', None, None, None, None, None, '2646451') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.021452903747558594 save_final save missing photos in datou_result : time spend for datou_step_exec : 14.578800439834595 time spend to save output : 0.021754026412963867 total time spend for step 9 : 14.600554466247559 step10:send_mail_cod Fri Mar 7 03:23:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Qualipapia_P21130892_07-03-2025_03_23_31.pdf 21133553 imagette211335531741314211 21133570 change filename to text .imagette211335701741314211 21133555 imagette211335551741314211 21133557 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette211335571741314211 21133558 imagette211335581741314213 21133560 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette211335601741314213 21133565 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette211335651741314213 21133567 imagette211335671741314214 21133563 imagette211335631741314214 21133566 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette211335661741314214 21133568 imagette211335681741314215 21133569 change filename to text .imagette211335691741314215 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=21130892 and hashtag_type = 4230 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/21133571,21133588,21133573,21133575,21133576,21133577,21133578,21133583,21133585,21133581,21133584,21133586,21133587?tags=tetrapak,pehd,flou,carton,film_plastique,environnement,autre,papier,pet_fonce,mal_croppe,pet_clair,refus,metal&datou_id_consolidate=4235&port_consolidate=21130892&tree_id=81841 your option no_mail is active, we will not send the real mail to your client args[1341329234] : ((1341329234, -8.248775659161165, 492609224), (1341329234, -0.17458670726997252, 496442774), '0.22029002069900358') We are sending mail with results at cod@fotonower.com args[1341329229] : ((1341329229, -8.236877990788066, 492609224), (1341329229, -0.1624521498548921, 496442774), '0.22029002069900358') We are sending mail with results at cod@fotonower.com refus_total : 0.22029002069900358 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21130892 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21130892_07-03-2025_03_23_31.pdf results_Qualipapia_P21130892_07-03-2025_03_23_31.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21130892_07-03-2025_03_23_31.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4234','21130892','results_Qualipapia_P21130892_07-03-2025_03_23_31.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21130892_07-03-2025_03_23_31.pdf','pdf','','0.15','0.22029002069900358') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1341329234, 1341329229] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2646451') ('4234', '21130892', '1341329234', None, None, None, None, None, '2646451') ('4234', None, None, None, None, None, None, None, '2646451') ('4234', '21130892', '1341329229', None, None, None, None, None, '2646451') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 2 time used for this insertion : 0.015036582946777344 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.9563212394714355 time spend to save output : 0.015229463577270508 total time spend for step 10 : 4.971550703048706 step11:split_time_score Fri Mar 7 03:23:36 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 3442, 'mtr_user_id': 31, 'name': 'classifieur_2camions_valcor_021122_v1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'deux_camions,camion_droite,camion_gauche,pas_de_camion', 'svm_portfolios_learning': '7659379,7659034,7657685,7657114', 'photo_hashtag_type': 4458, 'photo_desc_type': 5723, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107760533,2107760534,2107760535,2107760536'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('18', 2),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 06032025 21130892 Nombre de photos uploadées : 2 / 23040 (0%) 06032025 21130892 Nombre de photos taguées (types de déchets): 2 / 2 (100%) 06032025 21130892 Nombre de photos taguées (volume) : 0 / 2 (0%) elapsed_time : load_data_split_time_score 4.291534423828125e-06 elapsed_time : order_list_meta_photo_and_scores 2.002716064453125e-05 ?? elapsed_time : fill_and_build_computed_from_old_data 0.0002682209014892578 elapsed_time : insert_dashboard_record_day_entry 0.039223670959472656 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.2552810083254836 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21116418_06-03-2025_12_35_17.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21116418 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21116418 AND mptpi.`type`=4230 To do Qualite : 0.2971831971307864 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21116889_06-03-2025_13_32_43.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21116889 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21116889 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21116890 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21116891 order by id desc limit 1 Qualite : 0.2656426077944325 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21116892_06-03-2025_17_57_08.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21116892 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21116892 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21116893 order by id desc limit 1 Qualite : 0.2487335489167673 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21117391_06-03-2025_17_40_55.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21117391 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21117391 AND mptpi.`type`=4230 To do Qualite : 0.19606978781104878 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21121258_06-03-2025_19_33_40.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21121258 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21121258 AND mptpi.`type`=4230 To do Qualite : 0.24433758362757385 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21121259_06-03-2025_18_49_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21121259 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21121259 AND mptpi.`type`=4230 To do Qualite : 0.24199178619858192 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21132481_07-03-2025_02_19_07.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21132481 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21132481 AND mptpi.`type`=4230 To do Qualite : 0.257982925153035 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21131708_07-03-2025_01_49_40.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21131708 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21131708 AND mptpi.`type`=4230 To do Qualite : 0.2087070242602511 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21131709_07-03-2025_01_24_44.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21131709 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21131709 AND mptpi.`type`=4230 To do Qualite : 0.2294703936907165 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21131710_07-03-2025_01_36_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21131710 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21131710 AND mptpi.`type`=4230 To do Qualite : 0.20871812148057614 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21131713_07-03-2025_01_17_56.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21131713 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21131713 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130819 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130822 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130824 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130825 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130826 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130827 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130828 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21127124 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21127125 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21127127 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130829 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130830 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130848 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130878 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21127129 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21127131 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21127132 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21127134 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21127135 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21127137 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21127138 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21127139 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21127141 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21127142 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21127144 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21127145 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21127147 order by id desc limit 1 Qualite : 0.24016860385296235 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21127148_06-03-2025_21_35_14.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21127148 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21127148 AND mptpi.`type`=4230 To do Qualite : 0.24463049736262288 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21127150_06-03-2025_21_48_55.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21127150 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21127150 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21127151 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130879 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21129183 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21129184 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21129185 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21129186 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21129187 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21129188 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21129189 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21129190 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21129191 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21129192 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130880 order by id desc limit 1 Qualite : 0.21257227382097693 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21129194_06-03-2025_22_50_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21129194 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21129194 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21129195 order by id desc limit 1 Qualite : 0.16554427250971745 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21129196_06-03-2025_22_20_33.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21129196 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21129196 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130881 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130882 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130883 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130884 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130885 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130886 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21129605 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21129606 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21129607 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21129608 order by id desc limit 1 Qualite : 0.22426695870248817 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21129609_07-03-2025_00_02_02.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21129609 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21129609 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130887 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130888 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130889 order by id desc limit 1 Qualite : 0.25858088245874145 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21130890_07-03-2025_03_15_48.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130890 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21130890 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130891 order by id desc limit 1 Qualite : 0.22029002069900358 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21130892_07-03-2025_03_23_31.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130892 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21130892 AND mptpi.`type`=4230 To do Qualite : 0.20728644885268993 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21130893_07-03-2025_00_50_27.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130893 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21130893 AND mptpi.`type`=4230 To do Qualite : 0.22354729983925703 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21130894_07-03-2025_00_44_38.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130894 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21130894 AND mptpi.`type`=4230 To do Qualite : 0.22282342256478435 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21130895_07-03-2025_00_36_23.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21130895 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21130895 AND mptpi.`type`=4230 To do Qualite : 0.24537214151833436 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21131716_07-03-2025_01_11_57.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21131716 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21131716 AND mptpi.`type`=4230 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'06032025': {'nb_upload': 2, 'nb_taggue_class': 2, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1341329234, 1341329229] Looping around the photos to save general results len do output : 1 /21130892Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2646451') ('4234', '21130892', '1341329234', None, None, None, None, None, '2646451') ('4234', None, None, None, None, None, None, None, '2646451') ('4234', '21130892', '1341329229', None, None, None, None, None, '2646451') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.94277024269104 save_final save missing photos in datou_result : time spend for datou_step_exec : 48.57364320755005 time spend to save output : 0.9430055618286133 total time spend for step 11 : 49.51664876937866 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 1 len(input) + len(total_photo_id_missing) : 2 set_done_treatment 35.72user 24.15system 4:01.29elapsed 24%CPU (0avgtext+0avgdata 2499144maxresident)k 694784inputs+12112outputs (71major+2360440minor)pagefaults 0swaps