python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 4234 -l 5 ' -s datou_current_4234 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 4129763 load datou : 4234 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load THCL from format json or kwargs add thcl : 3237 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig add pdt : 5561 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4234, datou_cur_ids : ['2573411'] with mtr_portfolio_ids : ['20415723'] and first list_photo_ids : [] new path : /proc/4129763/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, brightness, blur_detection, rle_unique_nms_with_priority, crop_condition, thcl, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , BFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 2 ; length of list_pids : 2 ; length of list_args : 2 time to download the photos : 0.2661452293395996 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Mon Feb 10 21:20:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10553 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-10 21:20:31.849547: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-10 21:20:31.875055: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-10 21:20:31.877176: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fc08c000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-10 21:20:31.877241: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-10 21:20:31.881430: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-10 21:20:31.997878: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1e3049c0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-10 21:20:31.997922: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-10 21:20:31.999239: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-10 21:20:31.999681: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-10 21:20:32.002989: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-10 21:20:32.005779: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-10 21:20:32.006181: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-10 21:20:32.008831: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-10 21:20:32.010126: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-10 21:20:32.016515: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-10 21:20:32.017945: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-10 21:20:32.018020: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-10 21:20:32.018746: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-10 21:20:32.018761: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-10 21:20:32.018770: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-10 21:20:32.020069: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9633 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-10 21:20:32.328641: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-10 21:20:32.328772: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-10 21:20:32.328796: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-10 21:20:32.328817: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-10 21:20:32.328838: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-10 21:20:32.328858: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-10 21:20:32.328878: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-10 21:20:32.328899: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-10 21:20:32.330408: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-10 21:20:32.331783: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-10 21:20:32.331821: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-10 21:20:32.331839: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-10 21:20:32.331857: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-10 21:20:32.331873: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-10 21:20:32.331890: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-10 21:20:32.331907: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-10 21:20:32.331924: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-10 21:20:32.333419: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-10 21:20:32.333456: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-10 21:20:32.333467: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-10 21:20:32.333476: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-10 21:20:32.335088: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9633 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-10 21:20:41.231994: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-10 21:20:41.444286: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 2 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -116.27031 max: 146.61562 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 10 time for calcul the mask position with numpy : 8.845329284667969e-05 nb_pixel_total : 273 time to create 1 rle with old method : 0.0005807876586914062 length of segment : 36 time for calcul the mask position with numpy : 7.748603820800781e-05 nb_pixel_total : 49 time to create 1 rle with old method : 0.00018167495727539062 length of segment : 9 time for calcul the mask position with numpy : 8.249282836914062e-05 nb_pixel_total : 784 time to create 1 rle with old method : 0.0016181468963623047 length of segment : 31 time for calcul the mask position with numpy : 5.888938903808594e-05 nb_pixel_total : 224 time to create 1 rle with old method : 0.00048160552978515625 length of segment : 18 time for calcul the mask position with numpy : 6.461143493652344e-05 nb_pixel_total : 835 time to create 1 rle with old method : 0.0015718936920166016 length of segment : 33 time for calcul the mask position with numpy : 0.00010585784912109375 nb_pixel_total : 1342 time to create 1 rle with old method : 0.0027472972869873047 length of segment : 52 time for calcul the mask position with numpy : 5.698204040527344e-05 nb_pixel_total : 87 time to create 1 rle with old method : 0.0002334117889404297 length of segment : 26 time for calcul the mask position with numpy : 4.410743713378906e-05 nb_pixel_total : 5 time to create 1 rle with old method : 4.2438507080078125e-05 length of segment : 2 time for calcul the mask position with numpy : 6.413459777832031e-05 nb_pixel_total : 790 time to create 1 rle with old method : 0.0017125606536865234 length of segment : 35 time for calcul the mask position with numpy : 4.649162292480469e-05 nb_pixel_total : 19 time to create 1 rle with old method : 0.00014162063598632812 length of segment : 4 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.07500 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 4.7206878662109375e-05 nb_pixel_total : 920 time to create 1 rle with old method : 0.0012347698211669922 length of segment : 32 time for calcul the mask position with numpy : 4.291534423828125e-05 nb_pixel_total : 933 time to create 1 rle with old method : 0.0012087821960449219 length of segment : 30 time for calcul the mask position with numpy : 8.249282836914062e-05 nb_pixel_total : 2174 time to create 1 rle with old method : 0.0029180049896240234 length of segment : 89 time for calcul the mask position with numpy : 4.553794860839844e-05 nb_pixel_total : 890 time to create 1 rle with old method : 0.0011785030364990234 length of segment : 44 Processing 1 images image shape: (400, 400, 3) min: 7.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -100.89922 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 6.67572021484375e-05 nb_pixel_total : 1982 time to create 1 rle with old method : 0.0027074813842773438 length of segment : 36 time for calcul the mask position with numpy : 0.00011801719665527344 nb_pixel_total : 6432 time to create 1 rle with old method : 0.007577180862426758 length of segment : 113 time for calcul the mask position with numpy : 0.00013136863708496094 nb_pixel_total : 5300 time to create 1 rle with old method : 0.006453275680541992 length of segment : 96 time for calcul the mask position with numpy : 0.0004792213439941406 nb_pixel_total : 23936 time to create 1 rle with old method : 0.030896663665771484 length of segment : 340 time for calcul the mask position with numpy : 5.078315734863281e-05 nb_pixel_total : 579 time to create 1 rle with old method : 0.0007817745208740234 length of segment : 21 time for calcul the mask position with numpy : 5.507469177246094e-05 nb_pixel_total : 630 time to create 1 rle with old method : 0.0008950233459472656 length of segment : 20 time for calcul the mask position with numpy : 4.839897155761719e-05 nb_pixel_total : 1207 time to create 1 rle with old method : 0.0016224384307861328 length of segment : 36 time for calcul the mask position with numpy : 0.0001251697540283203 nb_pixel_total : 3227 time to create 1 rle with old method : 0.00399470329284668 length of segment : 116 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -122.68438 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 0.00010848045349121094 nb_pixel_total : 2205 time to create 1 rle with old method : 0.003281831741333008 length of segment : 177 time for calcul the mask position with numpy : 7.271766662597656e-05 nb_pixel_total : 252 time to create 1 rle with old method : 0.0005059242248535156 length of segment : 19 time for calcul the mask position with numpy : 6.580352783203125e-05 nb_pixel_total : 412 time to create 1 rle with old method : 0.0009338855743408203 length of segment : 23 time for calcul the mask position with numpy : 0.0002334117889404297 nb_pixel_total : 4650 time to create 1 rle with old method : 0.0076601505279541016 length of segment : 229 time for calcul the mask position with numpy : 5.054473876953125e-05 nb_pixel_total : 335 time to create 1 rle with old method : 0.0005753040313720703 length of segment : 23 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 253.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 139.58437 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 6.4849853515625e-05 nb_pixel_total : 541 time to create 1 rle with old method : 0.0008900165557861328 length of segment : 20 time for calcul the mask position with numpy : 7.176399230957031e-05 nb_pixel_total : 2445 time to create 1 rle with old method : 0.00315093994140625 length of segment : 159 time for calcul the mask position with numpy : 0.0006093978881835938 nb_pixel_total : 50997 time to create 1 rle with old method : 0.06682205200195312 length of segment : 266 Processing 1 images image shape: (400, 400, 3) min: 3.00000 max: 181.00000 molded_images shape: (1, 640, 640, 3) min: -93.46641 max: 47.03437 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -111.20000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 13 time for calcul the mask position with numpy : 8.487701416015625e-05 nb_pixel_total : 455 time to create 1 rle with old method : 0.0008389949798583984 length of segment : 22 time for calcul the mask position with numpy : 8.296966552734375e-05 nb_pixel_total : 1916 time to create 1 rle with old method : 0.0028874874114990234 length of segment : 34 time for calcul the mask position with numpy : 4.363059997558594e-05 nb_pixel_total : 257 time to create 1 rle with old method : 0.0006794929504394531 length of segment : 21 time for calcul the mask position with numpy : 6.127357482910156e-05 nb_pixel_total : 348 time to create 1 rle with old method : 0.0005729198455810547 length of segment : 18 time for calcul the mask position with numpy : 4.935264587402344e-05 nb_pixel_total : 130 time to create 1 rle with old method : 0.0002605915069580078 length of segment : 12 time for calcul the mask position with numpy : 5.936622619628906e-05 nb_pixel_total : 479 time to create 1 rle with old method : 0.0007572174072265625 length of segment : 43 time for calcul the mask position with numpy : 6.031990051269531e-05 nb_pixel_total : 194 time to create 1 rle with old method : 0.0003578662872314453 length of segment : 19 time for calcul the mask position with numpy : 5.7220458984375e-05 nb_pixel_total : 412 time to create 1 rle with old method : 0.0008642673492431641 length of segment : 46 time for calcul the mask position with numpy : 5.221366882324219e-05 nb_pixel_total : 111 time to create 1 rle with old method : 0.00022935867309570312 length of segment : 14 time for calcul the mask position with numpy : 4.76837158203125e-05 nb_pixel_total : 314 time to create 1 rle with old method : 0.0006544589996337891 length of segment : 24 time for calcul the mask position with numpy : 5.626678466796875e-05 nb_pixel_total : 341 time to create 1 rle with old method : 0.0006630420684814453 length of segment : 20 time for calcul the mask position with numpy : 5.817413330078125e-05 nb_pixel_total : 339 time to create 1 rle with old method : 0.0006036758422851562 length of segment : 20 time for calcul the mask position with numpy : 7.319450378417969e-05 nb_pixel_total : 1011 time to create 1 rle with old method : 0.0015392303466796875 length of segment : 41 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -106.90703 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 5.888938903808594e-05 nb_pixel_total : 763 time to create 1 rle with old method : 0.0012063980102539062 length of segment : 31 time for calcul the mask position with numpy : 3.743171691894531e-05 nb_pixel_total : 287 time to create 1 rle with old method : 0.0005080699920654297 length of segment : 13 time for calcul the mask position with numpy : 4.673004150390625e-05 nb_pixel_total : 655 time to create 1 rle with old method : 0.0009851455688476562 length of segment : 32 time for calcul the mask position with numpy : 7.939338684082031e-05 nb_pixel_total : 217 time to create 1 rle with old method : 0.0003502368927001953 length of segment : 54 time for calcul the mask position with numpy : 4.673004150390625e-05 nb_pixel_total : 596 time to create 1 rle with old method : 0.0010216236114501953 length of segment : 31 time for calcul the mask position with numpy : 5.221366882324219e-05 nb_pixel_total : 722 time to create 1 rle with old method : 0.001049041748046875 length of segment : 59 time for calcul the mask position with numpy : 4.291534423828125e-05 nb_pixel_total : 123 time to create 1 rle with old method : 0.00022459030151367188 length of segment : 19 time for calcul the mask position with numpy : 4.3392181396484375e-05 nb_pixel_total : 191 time to create 1 rle with old method : 0.00033664703369140625 length of segment : 20 time for calcul the mask position with numpy : 4.0531158447265625e-05 nb_pixel_total : 689 time to create 1 rle with old method : 0.0009546279907226562 length of segment : 30 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.30156 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 5.5789947509765625e-05 nb_pixel_total : 180 time to create 1 rle with old method : 0.0003218650817871094 length of segment : 16 time for calcul the mask position with numpy : 0.00015115737915039062 nb_pixel_total : 4392 time to create 1 rle with old method : 0.0057525634765625 length of segment : 231 time for calcul the mask position with numpy : 4.887580871582031e-05 nb_pixel_total : 735 time to create 1 rle with old method : 0.0010476112365722656 length of segment : 35 time for calcul the mask position with numpy : 3.4809112548828125e-05 nb_pixel_total : 217 time to create 1 rle with old method : 0.0003559589385986328 length of segment : 18 time for calcul the mask position with numpy : 0.00011491775512695312 nb_pixel_total : 1100 time to create 1 rle with old method : 0.0016853809356689453 length of segment : 112 time for calcul the mask position with numpy : 0.00011301040649414062 nb_pixel_total : 1401 time to create 1 rle with old method : 0.0025489330291748047 length of segment : 45 time for calcul the mask position with numpy : 3.790855407714844e-05 nb_pixel_total : 70 time to create 1 rle with old method : 0.00018739700317382812 length of segment : 11 time for calcul the mask position with numpy : 0.0002949237823486328 nb_pixel_total : 442 time to create 1 rle with old method : 0.0008971691131591797 length of segment : 79 time for calcul the mask position with numpy : 5.3882598876953125e-05 nb_pixel_total : 1389 time to create 1 rle with old method : 0.0019257068634033203 length of segment : 45 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 243.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 122.78359 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 4.267692565917969e-05 nb_pixel_total : 352 time to create 1 rle with old method : 0.0005364418029785156 length of segment : 36 time for calcul the mask position with numpy : 3.528594970703125e-05 nb_pixel_total : 353 time to create 1 rle with old method : 0.00045752525329589844 length of segment : 32 time for calcul the mask position with numpy : 4.220008850097656e-05 nb_pixel_total : 224 time to create 1 rle with old method : 0.0003657341003417969 length of segment : 20 time for calcul the mask position with numpy : 3.647804260253906e-05 nb_pixel_total : 375 time to create 1 rle with old method : 0.0005221366882324219 length of segment : 34 time for calcul the mask position with numpy : 0.00019693374633789062 nb_pixel_total : 4751 time to create 1 rle with old method : 0.0062999725341796875 length of segment : 220 Processing 1 images image shape: (280, 400, 3) min: 4.00000 max: 175.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 40.72734 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0010395050048828125 nb_pixel_total : 105777 time to create 1 rle with old method : 0.13548660278320312 length of segment : 281 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 16 time for calcul the mask position with numpy : 9.751319885253906e-05 nb_pixel_total : 3144 time to create 1 rle with old method : 0.004066944122314453 length of segment : 54 time for calcul the mask position with numpy : 6.437301635742188e-05 nb_pixel_total : 351 time to create 1 rle with old method : 0.0007596015930175781 length of segment : 23 time for calcul the mask position with numpy : 0.00013756752014160156 nb_pixel_total : 3211 time to create 1 rle with old method : 0.006251811981201172 length of segment : 108 time for calcul the mask position with numpy : 6.0558319091796875e-05 nb_pixel_total : 198 time to create 1 rle with old method : 0.00048661231994628906 length of segment : 9 time for calcul the mask position with numpy : 5.078315734863281e-05 nb_pixel_total : 401 time to create 1 rle with old method : 0.0008032321929931641 length of segment : 24 time for calcul the mask position with numpy : 7.295608520507812e-05 nb_pixel_total : 1385 time to create 1 rle with old method : 0.002650022506713867 length of segment : 37 time for calcul the mask position with numpy : 0.0001819133758544922 nb_pixel_total : 4841 time to create 1 rle with old method : 0.008606433868408203 length of segment : 158 time for calcul the mask position with numpy : 7.271766662597656e-05 nb_pixel_total : 464 time to create 1 rle with old method : 0.0009763240814208984 length of segment : 26 time for calcul the mask position with numpy : 6.437301635742188e-05 nb_pixel_total : 158 time to create 1 rle with old method : 0.0004315376281738281 length of segment : 9 time for calcul the mask position with numpy : 6.151199340820312e-05 nb_pixel_total : 413 time to create 1 rle with old method : 0.0008974075317382812 length of segment : 33 time for calcul the mask position with numpy : 4.172325134277344e-05 nb_pixel_total : 244 time to create 1 rle with old method : 0.005568981170654297 length of segment : 14 time for calcul the mask position with numpy : 8.177757263183594e-05 nb_pixel_total : 252 time to create 1 rle with old method : 0.0006206035614013672 length of segment : 9 time for calcul the mask position with numpy : 4.220008850097656e-05 nb_pixel_total : 168 time to create 1 rle with old method : 0.0003018379211425781 length of segment : 16 time for calcul the mask position with numpy : 5.364418029785156e-05 nb_pixel_total : 97 time to create 1 rle with old method : 0.0001556873321533203 length of segment : 16 time for calcul the mask position with numpy : 9.393692016601562e-05 nb_pixel_total : 1146 time to create 1 rle with old method : 0.0014836788177490234 length of segment : 55 time for calcul the mask position with numpy : 0.00017571449279785156 nb_pixel_total : 2446 time to create 1 rle with old method : 0.003041505813598633 length of segment : 51 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.77187 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 3.8623809814453125e-05 nb_pixel_total : 192 time to create 1 rle with old method : 0.00029659271240234375 length of segment : 13 time for calcul the mask position with numpy : 3.552436828613281e-05 nb_pixel_total : 170 time to create 1 rle with old method : 0.00026535987854003906 length of segment : 15 time for calcul the mask position with numpy : 3.0040740966796875e-05 nb_pixel_total : 60 time to create 1 rle with old method : 0.00011873245239257812 length of segment : 8 time for calcul the mask position with numpy : 3.361701965332031e-05 nb_pixel_total : 233 time to create 1 rle with old method : 0.00035190582275390625 length of segment : 18 time for calcul the mask position with numpy : 3.695487976074219e-05 nb_pixel_total : 331 time to create 1 rle with old method : 0.00045299530029296875 length of segment : 23 time for calcul the mask position with numpy : 3.170967102050781e-05 nb_pixel_total : 111 time to create 1 rle with old method : 0.00020503997802734375 length of segment : 10 time for calcul the mask position with numpy : 3.2901763916015625e-05 nb_pixel_total : 220 time to create 1 rle with old method : 0.0003476142883300781 length of segment : 16 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.06875 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 4.3392181396484375e-05 nb_pixel_total : 355 time to create 1 rle with old method : 0.0005004405975341797 length of segment : 22 time for calcul the mask position with numpy : 4.744529724121094e-05 nb_pixel_total : 352 time to create 1 rle with old method : 0.00047779083251953125 length of segment : 25 time for calcul the mask position with numpy : 3.528594970703125e-05 nb_pixel_total : 179 time to create 1 rle with old method : 0.0002815723419189453 length of segment : 17 time for calcul the mask position with numpy : 3.218650817871094e-05 nb_pixel_total : 136 time to create 1 rle with old method : 0.00022101402282714844 length of segment : 20 time for calcul the mask position with numpy : 3.0040740966796875e-05 nb_pixel_total : 87 time to create 1 rle with old method : 0.00015306472778320312 length of segment : 10 time for calcul the mask position with numpy : 4.172325134277344e-05 nb_pixel_total : 614 time to create 1 rle with old method : 0.0007874965667724609 length of segment : 32 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 138.72500 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 3.457069396972656e-05 nb_pixel_total : 15 time to create 1 rle with old method : 4.982948303222656e-05 length of segment : 3 time for calcul the mask position with numpy : 3.647804260253906e-05 nb_pixel_total : 586 time to create 1 rle with old method : 0.000766754150390625 length of segment : 33 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -116.93828 max: 145.36953 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 11 time for calcul the mask position with numpy : 8.249282836914062e-05 nb_pixel_total : 228 time to create 1 rle with old method : 0.0005154609680175781 length of segment : 35 time for calcul the mask position with numpy : 6.175041198730469e-05 nb_pixel_total : 204 time to create 1 rle with old method : 0.0004897117614746094 length of segment : 15 time for calcul the mask position with numpy : 6.127357482910156e-05 nb_pixel_total : 17 time to create 1 rle with old method : 9.441375732421875e-05 length of segment : 8 time for calcul the mask position with numpy : 6.151199340820312e-05 nb_pixel_total : 17 time to create 1 rle with old method : 0.00013566017150878906 length of segment : 3 time for calcul the mask position with numpy : 5.936622619628906e-05 nb_pixel_total : 822 time to create 1 rle with old method : 0.001544952392578125 length of segment : 30 time for calcul the mask position with numpy : 6.031990051269531e-05 nb_pixel_total : 717 time to create 1 rle with old method : 0.0013854503631591797 length of segment : 25 time for calcul the mask position with numpy : 4.172325134277344e-05 nb_pixel_total : 5 time to create 1 rle with old method : 4.4345855712890625e-05 length of segment : 2 time for calcul the mask position with numpy : 3.8623809814453125e-05 nb_pixel_total : 3 time to create 1 rle with old method : 2.765655517578125e-05 length of segment : 2 time for calcul the mask position with numpy : 3.5762786865234375e-05 nb_pixel_total : 19 time to create 1 rle with old method : 9.679794311523438e-05 length of segment : 4 time for calcul the mask position with numpy : 7.748603820800781e-05 nb_pixel_total : 45 time to create 1 rle with old method : 0.00012421607971191406 length of segment : 21 time for calcul the mask position with numpy : 9.131431579589844e-05 nb_pixel_total : 1964 time to create 1 rle with old method : 0.0036096572875976562 length of segment : 44 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -118.12969 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 4.935264587402344e-05 nb_pixel_total : 839 time to create 1 rle with old method : 0.0011415481567382812 length of segment : 40 time for calcul the mask position with numpy : 5.888938903808594e-05 nb_pixel_total : 985 time to create 1 rle with old method : 0.0012784004211425781 length of segment : 52 time for calcul the mask position with numpy : 4.553794860839844e-05 nb_pixel_total : 522 time to create 1 rle with old method : 0.0007262229919433594 length of segment : 56 time for calcul the mask position with numpy : 3.743171691894531e-05 nb_pixel_total : 50 time to create 1 rle with old method : 0.00010156631469726562 length of segment : 15 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -114.25469 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 0.00011110305786132812 nb_pixel_total : 6482 time to create 1 rle with old method : 0.007858991622924805 length of segment : 108 time for calcul the mask position with numpy : 8.893013000488281e-05 nb_pixel_total : 1729 time to create 1 rle with old method : 0.0024061203002929688 length of segment : 46 time for calcul the mask position with numpy : 0.0001404285430908203 nb_pixel_total : 5173 time to create 1 rle with old method : 0.006299495697021484 length of segment : 96 time for calcul the mask position with numpy : 6.985664367675781e-05 nb_pixel_total : 873 time to create 1 rle with old method : 0.0013625621795654297 length of segment : 31 time for calcul the mask position with numpy : 0.0006353855133056641 nb_pixel_total : 32031 time to create 1 rle with old method : 0.038091421127319336 length of segment : 410 time for calcul the mask position with numpy : 6.008148193359375e-05 nb_pixel_total : 1241 time to create 1 rle with old method : 0.0016524791717529297 length of segment : 34 time for calcul the mask position with numpy : 4.2438507080078125e-05 nb_pixel_total : 632 time to create 1 rle with old method : 0.0008959770202636719 length of segment : 23 time for calcul the mask position with numpy : 4.3392181396484375e-05 nb_pixel_total : 1137 time to create 1 rle with old method : 0.0015017986297607422 length of segment : 29 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 464 time to create 1 rle with old method : 0.0007402896881103516 length of segment : 26 time for calcul the mask position with numpy : 0.00011444091796875 nb_pixel_total : 2816 time to create 1 rle with old method : 0.0033316612243652344 length of segment : 212 time for calcul the mask position with numpy : 3.886222839355469e-05 nb_pixel_total : 380 time to create 1 rle with old method : 0.0005755424499511719 length of segment : 18 time for calcul the mask position with numpy : 3.4809112548828125e-05 nb_pixel_total : 327 time to create 1 rle with old method : 0.0004947185516357422 length of segment : 33 time for calcul the mask position with numpy : 3.147125244140625e-05 nb_pixel_total : 198 time to create 1 rle with old method : 0.00027632713317871094 length of segment : 22 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 140.53359 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 5.936622619628906e-05 nb_pixel_total : 563 time to create 1 rle with old method : 0.0010466575622558594 length of segment : 22 time for calcul the mask position with numpy : 0.0001888275146484375 nb_pixel_total : 2471 time to create 1 rle with old method : 0.00521540641784668 length of segment : 146 time for calcul the mask position with numpy : 8.96453857421875e-05 nb_pixel_total : 1396 time to create 1 rle with old method : 0.0022656917572021484 length of segment : 42 time for calcul the mask position with numpy : 0.000453948974609375 nb_pixel_total : 42125 time to create 1 rle with old method : 0.05449557304382324 length of segment : 192 Processing 1 images image shape: (400, 400, 3) min: 7.00000 max: 181.00000 molded_images shape: (1, 640, 640, 3) min: -89.09141 max: 46.47187 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -115.13359 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 15 time for calcul the mask position with numpy : 0.00010061264038085938 nb_pixel_total : 453 time to create 1 rle with old method : 0.0009598731994628906 length of segment : 22 time for calcul the mask position with numpy : 8.296966552734375e-05 nb_pixel_total : 1899 time to create 1 rle with old method : 0.0036935806274414062 length of segment : 32 time for calcul the mask position with numpy : 5.9604644775390625e-05 nb_pixel_total : 346 time to create 1 rle with old method : 0.00070953369140625 length of segment : 17 time for calcul the mask position with numpy : 7.152557373046875e-05 nb_pixel_total : 377 time to create 1 rle with old method : 0.0007107257843017578 length of segment : 31 time for calcul the mask position with numpy : 5.7697296142578125e-05 nb_pixel_total : 156 time to create 1 rle with old method : 0.00039005279541015625 length of segment : 13 time for calcul the mask position with numpy : 6.151199340820312e-05 nb_pixel_total : 384 time to create 1 rle with old method : 0.0007939338684082031 length of segment : 29 time for calcul the mask position with numpy : 7.653236389160156e-05 nb_pixel_total : 377 time to create 1 rle with old method : 0.0010352134704589844 length of segment : 18 time for calcul the mask position with numpy : 0.0003330707550048828 nb_pixel_total : 10432 time to create 1 rle with old method : 0.017882823944091797 length of segment : 131 time for calcul the mask position with numpy : 0.00010561943054199219 nb_pixel_total : 1041 time to create 1 rle with old method : 0.001970529556274414 length of segment : 42 time for calcul the mask position with numpy : 5.698204040527344e-05 nb_pixel_total : 124 time to create 1 rle with old method : 0.00031185150146484375 length of segment : 16 time for calcul the mask position with numpy : 6.604194641113281e-05 nb_pixel_total : 350 time to create 1 rle with old method : 0.0007712841033935547 length of segment : 27 time for calcul the mask position with numpy : 7.343292236328125e-05 nb_pixel_total : 229 time to create 1 rle with old method : 0.0005807876586914062 length of segment : 48 time for calcul the mask position with numpy : 5.888938903808594e-05 nb_pixel_total : 181 time to create 1 rle with old method : 0.0003724098205566406 length of segment : 20 time for calcul the mask position with numpy : 7.104873657226562e-05 nb_pixel_total : 250 time to create 1 rle with old method : 0.0005025863647460938 length of segment : 32 time for calcul the mask position with numpy : 5.9604644775390625e-05 nb_pixel_total : 350 time to create 1 rle with old method : 0.0006580352783203125 length of segment : 21 Processing 1 images image shape: (400, 400, 3) min: 1.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -105.46953 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 4.7206878662109375e-05 nb_pixel_total : 334 time to create 1 rle with old method : 0.0005078315734863281 length of segment : 14 time for calcul the mask position with numpy : 5.7697296142578125e-05 nb_pixel_total : 89 time to create 1 rle with old method : 0.0001850128173828125 length of segment : 17 time for calcul the mask position with numpy : 4.863739013671875e-05 nb_pixel_total : 220 time to create 1 rle with old method : 0.0003490447998046875 length of segment : 25 time for calcul the mask position with numpy : 5.7697296142578125e-05 nb_pixel_total : 812 time to create 1 rle with old method : 0.0012059211730957031 length of segment : 33 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 111 time to create 1 rle with old method : 0.0002288818359375 length of segment : 11 time for calcul the mask position with numpy : 6.818771362304688e-05 nb_pixel_total : 537 time to create 1 rle with old method : 0.0008039474487304688 length of segment : 29 time for calcul the mask position with numpy : 4.673004150390625e-05 nb_pixel_total : 204 time to create 1 rle with old method : 0.00029850006103515625 length of segment : 30 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -121.71953 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 5.316734313964844e-05 nb_pixel_total : 362 time to create 1 rle with old method : 0.0005562305450439453 length of segment : 19 time for calcul the mask position with numpy : 4.2438507080078125e-05 nb_pixel_total : 211 time to create 1 rle with old method : 0.0003299713134765625 length of segment : 16 time for calcul the mask position with numpy : 0.00017118453979492188 nb_pixel_total : 3834 time to create 1 rle with old method : 0.004932880401611328 length of segment : 193 time for calcul the mask position with numpy : 7.271766662597656e-05 nb_pixel_total : 1388 time to create 1 rle with old method : 0.0017955303192138672 length of segment : 42 time for calcul the mask position with numpy : 8.130073547363281e-05 nb_pixel_total : 1062 time to create 1 rle with old method : 0.0014193058013916016 length of segment : 84 time for calcul the mask position with numpy : 5.245208740234375e-05 nb_pixel_total : 848 time to create 1 rle with old method : 0.0011310577392578125 length of segment : 36 time for calcul the mask position with numpy : 0.00011706352233886719 nb_pixel_total : 1266 time to create 1 rle with old method : 0.001646280288696289 length of segment : 132 time for calcul the mask position with numpy : 0.00018835067749023438 nb_pixel_total : 4024 time to create 1 rle with old method : 0.005045890808105469 length of segment : 204 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 244.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 115.70547 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 5.7697296142578125e-05 nb_pixel_total : 840 time to create 1 rle with old method : 0.0011169910430908203 length of segment : 42 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 375 time to create 1 rle with old method : 0.0005109310150146484 length of segment : 35 time for calcul the mask position with numpy : 4.649162292480469e-05 nb_pixel_total : 340 time to create 1 rle with old method : 0.0004596710205078125 length of segment : 34 time for calcul the mask position with numpy : 7.104873657226562e-05 nb_pixel_total : 367 time to create 1 rle with old method : 0.0005779266357421875 length of segment : 26 time for calcul the mask position with numpy : 0.00020933151245117188 nb_pixel_total : 4986 time to create 1 rle with old method : 0.005995512008666992 length of segment : 221 time for calcul the mask position with numpy : 0.00020313262939453125 nb_pixel_total : 4342 time to create 1 rle with old method : 0.005248546600341797 length of segment : 192 Processing 1 images image shape: (280, 400, 3) min: 11.00000 max: 186.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 42.06562 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0011026859283447266 nb_pixel_total : 105010 time to create 1 rle with old method : 0.12238740921020508 length of segment : 281 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 15 time for calcul the mask position with numpy : 6.771087646484375e-05 nb_pixel_total : 479 time to create 1 rle with old method : 0.0006515979766845703 length of segment : 28 time for calcul the mask position with numpy : 4.172325134277344e-05 nb_pixel_total : 183 time to create 1 rle with old method : 0.00032520294189453125 length of segment : 9 time for calcul the mask position with numpy : 8.440017700195312e-05 nb_pixel_total : 3148 time to create 1 rle with old method : 0.0039212703704833984 length of segment : 56 time for calcul the mask position with numpy : 5.173683166503906e-05 nb_pixel_total : 449 time to create 1 rle with old method : 0.0006709098815917969 length of segment : 24 time for calcul the mask position with numpy : 8.463859558105469e-05 nb_pixel_total : 1857 time to create 1 rle with old method : 0.0024864673614501953 length of segment : 72 time for calcul the mask position with numpy : 9.846687316894531e-05 nb_pixel_total : 1377 time to create 1 rle with old method : 0.0028314590454101562 length of segment : 46 time for calcul the mask position with numpy : 7.82012939453125e-05 nb_pixel_total : 135 time to create 1 rle with old method : 0.00037932395935058594 length of segment : 12 time for calcul the mask position with numpy : 6.604194641113281e-05 nb_pixel_total : 373 time to create 1 rle with old method : 0.00081634521484375 length of segment : 22 time for calcul the mask position with numpy : 6.103515625e-05 nb_pixel_total : 191 time to create 1 rle with old method : 0.0005025863647460938 length of segment : 13 time for calcul the mask position with numpy : 8.678436279296875e-05 nb_pixel_total : 896 time to create 1 rle with old method : 0.001894235610961914 length of segment : 52 time for calcul the mask position with numpy : 0.0001461505889892578 nb_pixel_total : 258 time to create 1 rle with old method : 0.0007381439208984375 length of segment : 15 time for calcul the mask position with numpy : 6.818771362304688e-05 nb_pixel_total : 98 time to create 1 rle with old method : 0.0002465248107910156 length of segment : 17 time for calcul the mask position with numpy : 0.0002532005310058594 nb_pixel_total : 4156 time to create 1 rle with old method : 0.008530855178833008 length of segment : 133 time for calcul the mask position with numpy : 0.00021338462829589844 nb_pixel_total : 3032 time to create 1 rle with old method : 0.008265018463134766 length of segment : 100 time for calcul the mask position with numpy : 0.00015687942504882812 nb_pixel_total : 2206 time to create 1 rle with old method : 0.002967357635498047 length of segment : 57 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 7.43865966796875e-05 nb_pixel_total : 179 time to create 1 rle with old method : 0.00031065940856933594 length of segment : 14 time for calcul the mask position with numpy : 5.817413330078125e-05 nb_pixel_total : 239 time to create 1 rle with old method : 0.0005576610565185547 length of segment : 18 time for calcul the mask position with numpy : 3.790855407714844e-05 nb_pixel_total : 195 time to create 1 rle with old method : 0.00029659271240234375 length of segment : 13 time for calcul the mask position with numpy : 4.458427429199219e-05 nb_pixel_total : 369 time to create 1 rle with old method : 0.0005028247833251953 length of segment : 25 time for calcul the mask position with numpy : 3.361701965332031e-05 nb_pixel_total : 47 time to create 1 rle with old method : 9.512901306152344e-05 length of segment : 7 time for calcul the mask position with numpy : 3.5762786865234375e-05 nb_pixel_total : 230 time to create 1 rle with old method : 0.0004048347473144531 length of segment : 17 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.90859 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 4.57763671875e-05 nb_pixel_total : 103 time to create 1 rle with old method : 0.00018143653869628906 length of segment : 10 time for calcul the mask position with numpy : 4.0531158447265625e-05 nb_pixel_total : 378 time to create 1 rle with old method : 0.0005283355712890625 length of segment : 22 time for calcul the mask position with numpy : 5.841255187988281e-05 nb_pixel_total : 815 time to create 1 rle with old method : 0.0010495185852050781 length of segment : 47 time for calcul the mask position with numpy : 5.078315734863281e-05 nb_pixel_total : 305 time to create 1 rle with old method : 0.00044083595275878906 length of segment : 24 time for calcul the mask position with numpy : 4.1484832763671875e-05 nb_pixel_total : 165 time to create 1 rle with old method : 0.00029087066650390625 length of segment : 16 time for calcul the mask position with numpy : 7.557868957519531e-05 nb_pixel_total : 93 time to create 1 rle with old method : 0.0001628398895263672 length of segment : 21 time for calcul the mask position with numpy : 0.00012540817260742188 nb_pixel_total : 232 time to create 1 rle with old method : 0.00048613548278808594 length of segment : 47 time for calcul the mask position with numpy : 5.435943603515625e-05 nb_pixel_total : 388 time to create 1 rle with old method : 0.0006113052368164062 length of segment : 32 time for calcul the mask position with numpy : 5.6743621826171875e-05 nb_pixel_total : 1197 time to create 1 rle with old method : 0.0015375614166259766 length of segment : 75 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 132.97500 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.004187345504760742 nb_pixel_total : 32096 time to create 1 rle with old method : 0.06501531600952148 length of segment : 206 Detection mask done ! Trying to reset tf kernel 4130261 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 688 tf kernel not reseted sub process len(results) : 198 len(list_Values) 198 None max_time_sub_proc : 3600 parent process len(results) : 0 len(list_Values) 198 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5977 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Catched exception ! Connect or reconnect ! Number saved : None batch 1 Loaded 198 chid ids of type : 4228 Number RLEs to save : 10187 save missing photos in datou_result : time spend for datou_step_exec : 24.82161831855774 time spend to save output : 0.8325042724609375 total time spend for step 1 : 25.654122591018677 step2:brightness Mon Feb 10 21:20:54 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1739218828_4129763_1336527589_ad9179063831207b4b3aef8114938ecf.jpg treat image : temp/1739218828_4129763_1336527560_7b0c1fb6053f177c4b4683a9fe8d4995.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 2 time used for this insertion : 0.011785507202148438 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 2 time used for this insertion : 0.011988639831542969 save missing photos in datou_result : time spend for datou_step_exec : 0.8725953102111816 time spend to save output : 0.028059720993041992 total time spend for step 2 : 0.9006550312042236 step3:blur_detection Mon Feb 10 21:20:55 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1739218828_4129763_1336527589_ad9179063831207b4b3aef8114938ecf.jpg resize: (1080, 1920) 1336527589 -7.736193306868757 treat image : temp/1739218828_4129763_1336527560_7b0c1fb6053f177c4b4683a9fe8d4995.jpg resize: (1080, 1920) 1336527560 -7.738081110914254 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 2 time used for this insertion : 0.013794422149658203 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 2 time used for this insertion : 0.011787176132202148 save missing photos in datou_result : time spend for datou_step_exec : 1.9548218250274658 time spend to save output : 0.030471324920654297 total time spend for step 3 : 1.9852931499481201 step4:rle_unique_nms_with_priority Mon Feb 10 21:20:57 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 198 chid ids of type : 4228 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 98 nb_hashtags : 5 time to prepare the origin masks : 1.0983691215515137 time for calcul the mask position with numpy : 0.2410438060760498 nb_pixel_total : 1814397 time to create 1 rle with new method : 0.37146830558776855 time for calcul the mask position with numpy : 0.009693622589111328 nb_pixel_total : 23936 time to create 1 rle with old method : 0.027169227600097656 time for calcul the mask position with numpy : 0.008789300918579102 nb_pixel_total : 2445 time to create 1 rle with old method : 0.0029304027557373047 time for calcul the mask position with numpy : 0.00883626937866211 nb_pixel_total : 5300 time to create 1 rle with old method : 0.00612330436706543 time for calcul the mask position with numpy : 0.00862741470336914 nb_pixel_total : 224 time to create 1 rle with old method : 0.0002887248992919922 time for calcul the mask position with numpy : 0.008517026901245117 nb_pixel_total : 920 time to create 1 rle with old method : 0.0011239051818847656 time for calcul the mask position with numpy : 0.008498668670654297 nb_pixel_total : 48 time to create 1 rle with old method : 0.00011324882507324219 time for calcul the mask position with numpy : 0.008489847183227539 nb_pixel_total : 19 time to create 1 rle with old method : 7.748603820800781e-05 time for calcul the mask position with numpy : 0.008625030517578125 nb_pixel_total : 87 time to create 1 rle with old method : 0.00016188621520996094 time for calcul the mask position with numpy : 0.008528470993041992 nb_pixel_total : 74 time to create 1 rle with old method : 0.00011420249938964844 time for calcul the mask position with numpy : 0.00850367546081543 nb_pixel_total : 784 time to create 1 rle with old method : 0.0009610652923583984 time for calcul the mask position with numpy : 0.008586406707763672 nb_pixel_total : 49 time to create 1 rle with old method : 8.511543273925781e-05 time for calcul the mask position with numpy : 0.008617401123046875 nb_pixel_total : 273 time to create 1 rle with old method : 0.0003981590270996094 time for calcul the mask position with numpy : 0.009098529815673828 nb_pixel_total : 50997 time to create 1 rle with old method : 0.056548357009887695 time for calcul the mask position with numpy : 0.008665084838867188 nb_pixel_total : 5 time to create 1 rle with old method : 3.361701965332031e-05 time for calcul the mask position with numpy : 0.011018037796020508 nb_pixel_total : 3 time to create 1 rle with old method : 6.4849853515625e-05 time for calcul the mask position with numpy : 0.011710166931152344 nb_pixel_total : 11 time to create 1 rle with old method : 4.601478576660156e-05 time for calcul the mask position with numpy : 0.00868535041809082 nb_pixel_total : 2205 time to create 1 rle with old method : 0.0025026798248291016 time for calcul the mask position with numpy : 0.00862884521484375 nb_pixel_total : 252 time to create 1 rle with old method : 0.0003268718719482422 time for calcul the mask position with numpy : 0.008651494979858398 nb_pixel_total : 1342 time to create 1 rle with old method : 0.001645803451538086 time for calcul the mask position with numpy : 0.008591651916503906 nb_pixel_total : 335 time to create 1 rle with old method : 0.0004191398620605469 time for calcul the mask position with numpy : 0.008687734603881836 nb_pixel_total : 890 time to create 1 rle with old method : 0.0011289119720458984 time for calcul the mask position with numpy : 0.008627653121948242 nb_pixel_total : 579 time to create 1 rle with old method : 0.0007386207580566406 time for calcul the mask position with numpy : 0.008582592010498047 nb_pixel_total : 4650 time to create 1 rle with old method : 0.006672859191894531 time for calcul the mask position with numpy : 0.008483409881591797 nb_pixel_total : 240 time to create 1 rle with old method : 0.0003025531768798828 time for calcul the mask position with numpy : 0.008438348770141602 nb_pixel_total : 412 time to create 1 rle with old method : 0.0005221366882324219 time for calcul the mask position with numpy : 0.008428096771240234 nb_pixel_total : 1982 time to create 1 rle with old method : 0.002407550811767578 time for calcul the mask position with numpy : 0.008497953414916992 nb_pixel_total : 6432 time to create 1 rle with old method : 0.00756382942199707 time for calcul the mask position with numpy : 0.008465051651000977 nb_pixel_total : 2174 time to create 1 rle with old method : 0.0026252269744873047 time for calcul the mask position with numpy : 0.008518218994140625 nb_pixel_total : 1207 time to create 1 rle with old method : 0.0014407634735107422 time for calcul the mask position with numpy : 0.008420944213867188 nb_pixel_total : 541 time to create 1 rle with old method : 0.0006225109100341797 time for calcul the mask position with numpy : 0.008393287658691406 nb_pixel_total : 180 time to create 1 rle with old method : 0.0002219676971435547 time for calcul the mask position with numpy : 0.008429765701293945 nb_pixel_total : 689 time to create 1 rle with old method : 0.0008192062377929688 time for calcul the mask position with numpy : 0.008450508117675781 nb_pixel_total : 735 time to create 1 rle with old method : 0.0009052753448486328 time for calcul the mask position with numpy : 0.008395671844482422 nb_pixel_total : 442 time to create 1 rle with old method : 0.0005741119384765625 time for calcul the mask position with numpy : 0.008394002914428711 nb_pixel_total : 287 time to create 1 rle with old method : 0.0003592967987060547 time for calcul the mask position with numpy : 0.00841522216796875 nb_pixel_total : 217 time to create 1 rle with old method : 0.0002753734588623047 time for calcul the mask position with numpy : 0.008406877517700195 nb_pixel_total : 22 time to create 1 rle with old method : 4.8160552978515625e-05 time for calcul the mask position with numpy : 0.008437395095825195 nb_pixel_total : 1401 time to create 1 rle with old method : 0.0016620159149169922 time for calcul the mask position with numpy : 0.00838613510131836 nb_pixel_total : 224 time to create 1 rle with old method : 0.00027108192443847656 time for calcul the mask position with numpy : 0.009078741073608398 nb_pixel_total : 4392 time to create 1 rle with old method : 0.005138874053955078 time for calcul the mask position with numpy : 0.00841379165649414 nb_pixel_total : 352 time to create 1 rle with old method : 0.0004277229309082031 time for calcul the mask position with numpy : 0.00829005241394043 nb_pixel_total : 4751 time to create 1 rle with old method : 0.005587100982666016 time for calcul the mask position with numpy : 0.008356571197509766 nb_pixel_total : 655 time to create 1 rle with old method : 0.0007724761962890625 time for calcul the mask position with numpy : 0.008367538452148438 nb_pixel_total : 1100 time to create 1 rle with old method : 0.0013163089752197266 time for calcul the mask position with numpy : 0.008391141891479492 nb_pixel_total : 1011 time to create 1 rle with old method : 0.001234292984008789 time for calcul the mask position with numpy : 0.008360147476196289 nb_pixel_total : 763 time to create 1 rle with old method : 0.0009555816650390625 time for calcul the mask position with numpy : 0.008385658264160156 nb_pixel_total : 722 time to create 1 rle with old method : 0.0008273124694824219 time for calcul the mask position with numpy : 0.008392333984375 nb_pixel_total : 528 time to create 1 rle with old method : 0.0006682872772216797 time for calcul the mask position with numpy : 0.008389949798583984 nb_pixel_total : 123 time to create 1 rle with old method : 0.00015974044799804688 time for calcul the mask position with numpy : 0.0083770751953125 nb_pixel_total : 51 time to create 1 rle with old method : 7.557868957519531e-05 time for calcul the mask position with numpy : 0.008402824401855469 nb_pixel_total : 353 time to create 1 rle with old method : 0.00041794776916503906 time for calcul the mask position with numpy : 0.0084075927734375 nb_pixel_total : 217 time to create 1 rle with old method : 0.00026702880859375 time for calcul the mask position with numpy : 0.00850057601928711 nb_pixel_total : 70 time to create 1 rle with old method : 0.00011110305786132812 time for calcul the mask position with numpy : 0.008484840393066406 nb_pixel_total : 110 time to create 1 rle with old method : 0.00014925003051757812 time for calcul the mask position with numpy : 0.008454561233520508 nb_pixel_total : 412 time to create 1 rle with old method : 0.0005185604095458984 time for calcul the mask position with numpy : 0.008451700210571289 nb_pixel_total : 215 time to create 1 rle with old method : 0.0002815723419189453 time for calcul the mask position with numpy : 0.008427143096923828 nb_pixel_total : 257 time to create 1 rle with old method : 0.00031685829162597656 time for calcul the mask position with numpy : 0.008430719375610352 nb_pixel_total : 130 time to create 1 rle with old method : 0.0001742839813232422 time for calcul the mask position with numpy : 0.009238958358764648 nb_pixel_total : 1916 time to create 1 rle with old method : 0.0028553009033203125 time for calcul the mask position with numpy : 0.009531021118164062 nb_pixel_total : 184 time to create 1 rle with old method : 0.00023651123046875 time for calcul the mask position with numpy : 0.009719610214233398 nb_pixel_total : 455 time to create 1 rle with old method : 0.0007038116455078125 time for calcul the mask position with numpy : 0.009439468383789062 nb_pixel_total : 479 time to create 1 rle with old method : 0.0005943775177001953 time for calcul the mask position with numpy : 0.008540630340576172 nb_pixel_total : 341 time to create 1 rle with old method : 0.0004253387451171875 time for calcul the mask position with numpy : 0.008533000946044922 nb_pixel_total : 53 time to create 1 rle with old method : 8.654594421386719e-05 time for calcul the mask position with numpy : 0.008557796478271484 nb_pixel_total : 348 time to create 1 rle with old method : 0.0004267692565917969 time for calcul the mask position with numpy : 0.008552312850952148 nb_pixel_total : 191 time to create 1 rle with old method : 0.00025177001953125 time for calcul the mask position with numpy : 0.010710954666137695 nb_pixel_total : 105777 time to create 1 rle with old method : 0.12623023986816406 time for calcul the mask position with numpy : 0.009778261184692383 nb_pixel_total : 198 time to create 1 rle with old method : 0.00035643577575683594 time for calcul the mask position with numpy : 0.009720087051391602 nb_pixel_total : 158 time to create 1 rle with old method : 0.0002894401550292969 time for calcul the mask position with numpy : 0.009781837463378906 nb_pixel_total : 53 time to create 1 rle with old method : 0.00011277198791503906 time for calcul the mask position with numpy : 0.009887218475341797 nb_pixel_total : 3144 time to create 1 rle with old method : 0.0051212310791015625 time for calcul the mask position with numpy : 0.009841442108154297 nb_pixel_total : 413 time to create 1 rle with old method : 0.0007522106170654297 time for calcul the mask position with numpy : 0.009416341781616211 nb_pixel_total : 170 time to create 1 rle with old method : 0.00027251243591308594 time for calcul the mask position with numpy : 0.008723735809326172 nb_pixel_total : 192 time to create 1 rle with old method : 0.0002589225769042969 time for calcul the mask position with numpy : 0.008691787719726562 nb_pixel_total : 1385 time to create 1 rle with old method : 0.0016951560974121094 time for calcul the mask position with numpy : 0.008693933486938477 nb_pixel_total : 4084 time to create 1 rle with old method : 0.005450248718261719 time for calcul the mask position with numpy : 0.008602142333984375 nb_pixel_total : 244 time to create 1 rle with old method : 0.00032401084899902344 time for calcul the mask position with numpy : 0.008548736572265625 nb_pixel_total : 168 time to create 1 rle with old method : 0.0002510547637939453 time for calcul the mask position with numpy : 0.008776664733886719 nb_pixel_total : 3211 time to create 1 rle with old method : 0.0048139095306396484 time for calcul the mask position with numpy : 0.008693218231201172 nb_pixel_total : 464 time to create 1 rle with old method : 0.0005915164947509766 time for calcul the mask position with numpy : 0.008594989776611328 nb_pixel_total : 97 time to create 1 rle with old method : 0.00014781951904296875 time for calcul the mask position with numpy : 0.008701324462890625 nb_pixel_total : 355 time to create 1 rle with old method : 0.0004620552062988281 time for calcul the mask position with numpy : 0.00856924057006836 nb_pixel_total : 389 time to create 1 rle with old method : 0.0005121231079101562 time for calcul the mask position with numpy : 0.008718252182006836 nb_pixel_total : 111 time to create 1 rle with old method : 0.0001857280731201172 time for calcul the mask position with numpy : 0.008677005767822266 nb_pixel_total : 179 time to create 1 rle with old method : 0.0002465248107910156 time for calcul the mask position with numpy : 0.00853729248046875 nb_pixel_total : 352 time to create 1 rle with old method : 0.000461578369140625 time for calcul the mask position with numpy : 0.008660078048706055 nb_pixel_total : 586 time to create 1 rle with old method : 0.0007295608520507812 time for calcul the mask position with numpy : 0.008723020553588867 nb_pixel_total : 15 time to create 1 rle with old method : 5.5789947509765625e-05 time for calcul the mask position with numpy : 0.008680582046508789 nb_pixel_total : 614 time to create 1 rle with old method : 0.0008041858673095703 time for calcul the mask position with numpy : 0.008701801300048828 nb_pixel_total : 2446 time to create 1 rle with old method : 0.0029184818267822266 time for calcul the mask position with numpy : 0.008440494537353516 nb_pixel_total : 233 time to create 1 rle with old method : 0.00028967857360839844 time for calcul the mask position with numpy : 0.008455038070678711 nb_pixel_total : 7 time to create 1 rle with old method : 3.361701965332031e-05 time for calcul the mask position with numpy : 0.00840449333190918 nb_pixel_total : 87 time to create 1 rle with old method : 0.00011181831359863281 time for calcul the mask position with numpy : 0.008425235748291016 nb_pixel_total : 401 time to create 1 rle with old method : 0.0004868507385253906 time for calcul the mask position with numpy : 0.008463621139526367 nb_pixel_total : 60 time to create 1 rle with old method : 8.249282836914062e-05 time for calcul the mask position with numpy : 0.008401632308959961 nb_pixel_total : 136 time to create 1 rle with old method : 0.00017309188842773438 time for calcul the mask position with numpy : 0.008417129516601562 nb_pixel_total : 331 time to create 1 rle with old method : 0.0006153583526611328 time for calcul the mask position with numpy : 0.00850367546081543 nb_pixel_total : 351 time to create 1 rle with old method : 0.00044226646423339844 create new chi : 1.793022871017456 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.015105485916137695 batch 1 Loaded 99 chid ids of type : 4230 Number RLEs to save : 10103 TO DO : save crop sub photo not yet done ! save time : 0.6019415855407715 nb_obj : 100 nb_hashtags : 6 time to prepare the origin masks : 2.1207966804504395 time for calcul the mask position with numpy : 0.2535836696624756 nb_pixel_total : 1776452 time to create 1 rle with new method : 0.2214832305908203 time for calcul the mask position with numpy : 0.009571552276611328 nb_pixel_total : 32031 time to create 1 rle with old method : 0.0451962947845459 time for calcul the mask position with numpy : 0.00864410400390625 nb_pixel_total : 5173 time to create 1 rle with old method : 0.007123708724975586 time for calcul the mask position with numpy : 0.007914304733276367 nb_pixel_total : 2471 time to create 1 rle with old method : 0.003778219223022461 time for calcul the mask position with numpy : 0.008232831954956055 nb_pixel_total : 204 time to create 1 rle with old method : 0.0002918243408203125 time for calcul the mask position with numpy : 0.00802922248840332 nb_pixel_total : 839 time to create 1 rle with old method : 0.0010831356048583984 time for calcul the mask position with numpy : 0.006955862045288086 nb_pixel_total : 42 time to create 1 rle with old method : 7.43865966796875e-05 time for calcul the mask position with numpy : 0.006831169128417969 nb_pixel_total : 822 time to create 1 rle with old method : 0.0009963512420654297 time for calcul the mask position with numpy : 0.0068950653076171875 nb_pixel_total : 1964 time to create 1 rle with old method : 0.002345561981201172 time for calcul the mask position with numpy : 0.006563663482666016 nb_pixel_total : 17 time to create 1 rle with old method : 4.4345855712890625e-05 time for calcul the mask position with numpy : 0.006924152374267578 nb_pixel_total : 228 time to create 1 rle with old method : 0.0003848075866699219 time for calcul the mask position with numpy : 0.0072479248046875 nb_pixel_total : 45 time to create 1 rle with old method : 0.00010919570922851562 time for calcul the mask position with numpy : 0.007108211517333984 nb_pixel_total : 50 time to create 1 rle with old method : 8.654594421386719e-05 time for calcul the mask position with numpy : 0.007399082183837891 nb_pixel_total : 16 time to create 1 rle with old method : 5.245208740234375e-05 time for calcul the mask position with numpy : 0.00713801383972168 nb_pixel_total : 5 time to create 1 rle with old method : 3.814697265625e-05 time for calcul the mask position with numpy : 0.0071926116943359375 nb_pixel_total : 35 time to create 1 rle with old method : 0.00012350082397460938 time for calcul the mask position with numpy : 0.009357929229736328 nb_pixel_total : 42125 time to create 1 rle with old method : 0.052561044692993164 time for calcul the mask position with numpy : 0.00674128532409668 nb_pixel_total : 2816 time to create 1 rle with old method : 0.0034339427947998047 time for calcul the mask position with numpy : 0.007350921630859375 nb_pixel_total : 16 time to create 1 rle with old method : 5.698204040527344e-05 time for calcul the mask position with numpy : 0.007315874099731445 nb_pixel_total : 3 time to create 1 rle with old method : 2.7179718017578125e-05 time for calcul the mask position with numpy : 0.007817268371582031 nb_pixel_total : 327 time to create 1 rle with old method : 0.0004181861877441406 time for calcul the mask position with numpy : 0.011027336120605469 nb_pixel_total : 985 time to create 1 rle with old method : 0.001192331314086914 time for calcul the mask position with numpy : 0.011561393737792969 nb_pixel_total : 198 time to create 1 rle with old method : 0.0002493858337402344 time for calcul the mask position with numpy : 0.006883144378662109 nb_pixel_total : 873 time to create 1 rle with old method : 0.0011250972747802734 time for calcul the mask position with numpy : 0.00712275505065918 nb_pixel_total : 1729 time to create 1 rle with old method : 0.002070188522338867 time for calcul the mask position with numpy : 0.0069730281829833984 nb_pixel_total : 226 time to create 1 rle with old method : 0.0002956390380859375 time for calcul the mask position with numpy : 0.007026195526123047 nb_pixel_total : 464 time to create 1 rle with old method : 0.0005772113800048828 time for calcul the mask position with numpy : 0.008081912994384766 nb_pixel_total : 380 time to create 1 rle with old method : 0.00048351287841796875 time for calcul the mask position with numpy : 0.006887912750244141 nb_pixel_total : 6482 time to create 1 rle with old method : 0.007693767547607422 time for calcul the mask position with numpy : 0.007889509201049805 nb_pixel_total : 1396 time to create 1 rle with old method : 0.0017046928405761719 time for calcul the mask position with numpy : 0.007828712463378906 nb_pixel_total : 1241 time to create 1 rle with old method : 0.001497507095336914 time for calcul the mask position with numpy : 0.007378101348876953 nb_pixel_total : 65 time to create 1 rle with old method : 0.00010251998901367188 time for calcul the mask position with numpy : 0.007501840591430664 nb_pixel_total : 563 time to create 1 rle with old method : 0.0006916522979736328 time for calcul the mask position with numpy : 0.007027864456176758 nb_pixel_total : 211 time to create 1 rle with old method : 0.00027441978454589844 time for calcul the mask position with numpy : 0.006591796875 nb_pixel_total : 848 time to create 1 rle with old method : 0.0009772777557373047 time for calcul the mask position with numpy : 0.0068628787994384766 nb_pixel_total : 334 time to create 1 rle with old method : 0.0004284381866455078 time for calcul the mask position with numpy : 0.0075225830078125 nb_pixel_total : 1388 time to create 1 rle with old method : 0.001664876937866211 time for calcul the mask position with numpy : 0.007654428482055664 nb_pixel_total : 10055 time to create 1 rle with old method : 0.01209402084350586 time for calcul the mask position with numpy : 0.0067043304443359375 nb_pixel_total : 111 time to create 1 rle with old method : 0.00016832351684570312 time for calcul the mask position with numpy : 0.006868839263916016 nb_pixel_total : 840 time to create 1 rle with old method : 0.0010378360748291016 time for calcul the mask position with numpy : 0.008307695388793945 nb_pixel_total : 254 time to create 1 rle with old method : 0.00034999847412109375 time for calcul the mask position with numpy : 0.006759166717529297 nb_pixel_total : 3834 time to create 1 rle with old method : 0.004432201385498047 time for calcul the mask position with numpy : 0.007296085357666016 nb_pixel_total : 367 time to create 1 rle with old method : 0.0004584789276123047 time for calcul the mask position with numpy : 0.006669044494628906 nb_pixel_total : 4986 time to create 1 rle with old method : 0.0059356689453125 time for calcul the mask position with numpy : 0.006922721862792969 nb_pixel_total : 1062 time to create 1 rle with old method : 0.0012791156768798828 time for calcul the mask position with numpy : 0.006901264190673828 nb_pixel_total : 537 time to create 1 rle with old method : 0.0007119178771972656 time for calcul the mask position with numpy : 0.0072498321533203125 nb_pixel_total : 1266 time to create 1 rle with old method : 0.001575469970703125 time for calcul the mask position with numpy : 0.007013082504272461 nb_pixel_total : 812 time to create 1 rle with old method : 0.0009920597076416016 time for calcul the mask position with numpy : 0.00652003288269043 nb_pixel_total : 68 time to create 1 rle with old method : 0.000118255615234375 time for calcul the mask position with numpy : 0.006459474563598633 nb_pixel_total : 215 time to create 1 rle with old method : 0.0003235340118408203 time for calcul the mask position with numpy : 0.006748199462890625 nb_pixel_total : 377 time to create 1 rle with old method : 0.000499725341796875 time for calcul the mask position with numpy : 0.006747722625732422 nb_pixel_total : 89 time to create 1 rle with old method : 0.00013184547424316406 time for calcul the mask position with numpy : 0.006712675094604492 nb_pixel_total : 181 time to create 1 rle with old method : 0.0002353191375732422 time for calcul the mask position with numpy : 0.007096529006958008 nb_pixel_total : 375 time to create 1 rle with old method : 0.0006387233734130859 time for calcul the mask position with numpy : 0.006851911544799805 nb_pixel_total : 1 time to create 1 rle with old method : 2.86102294921875e-05 time for calcul the mask position with numpy : 0.0069200992584228516 nb_pixel_total : 362 time to create 1 rle with old method : 0.0006868839263916016 time for calcul the mask position with numpy : 0.006821632385253906 nb_pixel_total : 124 time to create 1 rle with old method : 0.0002357959747314453 time for calcul the mask position with numpy : 0.006818294525146484 nb_pixel_total : 229 time to create 1 rle with old method : 0.00042319297790527344 time for calcul the mask position with numpy : 0.0066835880279541016 nb_pixel_total : 377 time to create 1 rle with old method : 0.0006394386291503906 time for calcul the mask position with numpy : 0.006813764572143555 nb_pixel_total : 236 time to create 1 rle with old method : 0.0004372596740722656 time for calcul the mask position with numpy : 0.006656169891357422 nb_pixel_total : 222 time to create 1 rle with old method : 0.0004146099090576172 time for calcul the mask position with numpy : 0.006697654724121094 nb_pixel_total : 156 time to create 1 rle with old method : 0.00029540061950683594 time for calcul the mask position with numpy : 0.00680994987487793 nb_pixel_total : 204 time to create 1 rle with old method : 0.00036025047302246094 time for calcul the mask position with numpy : 0.007264614105224609 nb_pixel_total : 1899 time to create 1 rle with old method : 0.003263235092163086 time for calcul the mask position with numpy : 0.007657289505004883 nb_pixel_total : 453 time to create 1 rle with old method : 0.0006947517395019531 time for calcul the mask position with numpy : 0.009538650512695312 nb_pixel_total : 220 time to create 1 rle with old method : 0.0002906322479248047 time for calcul the mask position with numpy : 0.009462833404541016 nb_pixel_total : 384 time to create 1 rle with old method : 0.0004942417144775391 time for calcul the mask position with numpy : 0.009366512298583984 nb_pixel_total : 339 time to create 1 rle with old method : 0.0004189014434814453 time for calcul the mask position with numpy : 0.009190082550048828 nb_pixel_total : 346 time to create 1 rle with old method : 0.0004754066467285156 time for calcul the mask position with numpy : 0.010169029235839844 nb_pixel_total : 105010 time to create 1 rle with old method : 0.11736583709716797 time for calcul the mask position with numpy : 0.00996851921081543 nb_pixel_total : 191 time to create 1 rle with old method : 0.00025081634521484375 time for calcul the mask position with numpy : 0.009731531143188477 nb_pixel_total : 183 time to create 1 rle with old method : 0.00023818016052246094 time for calcul the mask position with numpy : 0.009732246398925781 nb_pixel_total : 3148 time to create 1 rle with old method : 0.003580808639526367 time for calcul the mask position with numpy : 0.009165048599243164 nb_pixel_total : 896 time to create 1 rle with old method : 0.0010967254638671875 time for calcul the mask position with numpy : 0.009316205978393555 nb_pixel_total : 179 time to create 1 rle with old method : 0.0002346038818359375 time for calcul the mask position with numpy : 0.009105205535888672 nb_pixel_total : 1377 time to create 1 rle with old method : 0.0015773773193359375 time for calcul the mask position with numpy : 0.009342432022094727 nb_pixel_total : 195 time to create 1 rle with old method : 0.0002524852752685547 time for calcul the mask position with numpy : 0.010204076766967773 nb_pixel_total : 32096 time to create 1 rle with old method : 0.035677194595336914 time for calcul the mask position with numpy : 0.009348392486572266 nb_pixel_total : 258 time to create 1 rle with old method : 0.0003311634063720703 time for calcul the mask position with numpy : 0.006857395172119141 nb_pixel_total : 1857 time to create 1 rle with old method : 0.0022537708282470703 time for calcul the mask position with numpy : 0.010621070861816406 nb_pixel_total : 116 time to create 1 rle with old method : 0.00016117095947265625 time for calcul the mask position with numpy : 0.01062917709350586 nb_pixel_total : 3032 time to create 1 rle with old method : 0.003629922866821289 time for calcul the mask position with numpy : 0.010606527328491211 nb_pixel_total : 479 time to create 1 rle with old method : 0.0005705356597900391 time for calcul the mask position with numpy : 0.011853218078613281 nb_pixel_total : 98 time to create 1 rle with old method : 0.00014209747314453125 time for calcul the mask position with numpy : 0.010851383209228516 nb_pixel_total : 378 time to create 1 rle with old method : 0.0004782676696777344 time for calcul the mask position with numpy : 0.011074304580688477 nb_pixel_total : 2010 time to create 1 rle with old method : 0.0023915767669677734 time for calcul the mask position with numpy : 0.010599613189697266 nb_pixel_total : 93 time to create 1 rle with old method : 0.00013947486877441406 time for calcul the mask position with numpy : 0.010570526123046875 nb_pixel_total : 4156 time to create 1 rle with old method : 0.004891872406005859 time for calcul the mask position with numpy : 0.010529279708862305 nb_pixel_total : 165 time to create 1 rle with old method : 0.0002300739288330078 time for calcul the mask position with numpy : 0.010344505310058594 nb_pixel_total : 305 time to create 1 rle with old method : 0.0003788471221923828 time for calcul the mask position with numpy : 0.01070261001586914 nb_pixel_total : 286 time to create 1 rle with old method : 0.0003590583801269531 time for calcul the mask position with numpy : 0.010576725006103516 nb_pixel_total : 815 time to create 1 rle with old method : 0.0010004043579101562 time for calcul the mask position with numpy : 0.011377811431884766 nb_pixel_total : 417 time to create 1 rle with old method : 0.0005342960357666016 time for calcul the mask position with numpy : 0.010801315307617188 nb_pixel_total : 232 time to create 1 rle with old method : 0.0003371238708496094 time for calcul the mask position with numpy : 0.010981321334838867 nb_pixel_total : 239 time to create 1 rle with old method : 0.0003058910369873047 time for calcul the mask position with numpy : 0.010600090026855469 nb_pixel_total : 103 time to create 1 rle with old method : 0.0001399517059326172 time for calcul the mask position with numpy : 0.00726008415222168 nb_pixel_total : 3 time to create 1 rle with old method : 3.1948089599609375e-05 time for calcul the mask position with numpy : 0.006961822509765625 nb_pixel_total : 373 time to create 1 rle with old method : 0.0004394054412841797 time for calcul the mask position with numpy : 0.008517980575561523 nb_pixel_total : 47 time to create 1 rle with old method : 7.390975952148438e-05 time for calcul the mask position with numpy : 0.009143352508544922 nb_pixel_total : 369 time to create 1 rle with old method : 0.0005462169647216797 time for calcul the mask position with numpy : 0.008853435516357422 nb_pixel_total : 449 time to create 1 rle with old method : 0.0005397796630859375 create new chi : 1.6777150630950928 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.0013439655303955078 batch 1 Loaded 101 chid ids of type : 4230 Number RLEs to save : 10779 TO DO : save crop sub photo not yet done ! save time : 0.6455898284912109 map_output_result : {1336527589: (0.0, 'Should be the crop_list due to order', 0.0), 1336527560: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1336527589, 1336527560] Looping around the photos to save general results len do output : 2 /1336527589.Didn't retrieve data . /1336527560.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2573411') ('4234', '20415723', '1336527589', None, None, None, None, None, '2573411') ('4234', None, None, None, None, None, None, None, '2573411') ('4234', '20415723', '1336527560', None, None, None, None, None, '2573411') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.012993335723876953 save_final save missing photos in datou_result : time spend for datou_step_exec : 8.38844347000122 time spend to save output : 0.013485431671142578 total time spend for step 4 : 8.401928901672363 step5:crop_condition Mon Feb 10 21:21:06 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4230 Loading chi in step crop for list_pids : 2 ! batch 1 Loaded 200 chid ids of type : 4230 begin to crop the class : papier param for this class : {'min_score': 0.6} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 77 About to insert : list_path_to_insert length 77 new photo from crops ! About to upload 77 photos upload in portfolio : 4869462 init cache_photo without model_param we have 77 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739218868_4129763 we have uploaded 77 photos in the portfolio 4869462 time of upload the photos Elapsed time : 16.85728621482849 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.6} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 21 About to insert : list_path_to_insert length 21 new photo from crops ! About to upload 21 photos upload in portfolio : 4869462 init cache_photo without model_param we have 21 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739218885_4129763 we have uploaded 21 photos in the portfolio 4869462 time of upload the photos Elapsed time : 4.766218900680542 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.6} filtre for class : metal hashtag_id of this class : 492628673 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739218890_4129763 we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.4984474182128906 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.6} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! About to upload 7 photos upload in portfolio : 4869462 init cache_photo without model_param we have 7 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739218890_4129763 we have uploaded 7 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.20489239692688 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.6} filtre for class : autre hashtag_id of this class : 494826614 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 33 About to insert : list_path_to_insert length 33 new photo from crops ! About to upload 33 photos upload in portfolio : 4869462 init cache_photo without model_param we have 33 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739218893_4129763 we have uploaded 33 photos in the portfolio 4869462 time of upload the photos Elapsed time : 7.269937992095947 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.6} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.6} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1336527589, 1336527560] Looping around the photos to save general results len do output : 139 /1336590338Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590339Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590340Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590341Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590342Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590343Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590344Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590345Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590346Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590347Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590348Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590349Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590350Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590351Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590352Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590353Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590354Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590356Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590357Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590358Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590359Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590360Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590361Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590362Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590363Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590364Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590365Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590366Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590367Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590368Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590369Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590370Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590371Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590372Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590373Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590374Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590375Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590376Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590377Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590378Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590379Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590380Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590381Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590382Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590383Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590384Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590385Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590386Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590387Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590388Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590389Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590390Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590391Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590392Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590393Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590394Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590395Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590396Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590397Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590398Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590399Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590400Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590401Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590402Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590403Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590404Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590405Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590406Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590407Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590408Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590409Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590410Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590411Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590412Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590413Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590414Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590415Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590418Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590419Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590420Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590421Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590422Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590423Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590424Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590425Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590426Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590427Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590428Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590429Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590430Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590431Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590432Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590433Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590434Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590435Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590436Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590437Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590438Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590440Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590442Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590443Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590444Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590445Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590446Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590447Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590448Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590455Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590459Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590463Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590467Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590471Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590475Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590479Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590483Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590486Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590489Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590493Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590497Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590501Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590505Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590509Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590512Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590516Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590520Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590524Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590528Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590532Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590536Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590539Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590543Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590547Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590551Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590555Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590559Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590563Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590567Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590571Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590575Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336590579Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2573411') ('4234', '20415723', '1336527589', None, None, None, None, None, '2573411') ('4234', None, None, None, None, None, None, None, '2573411') ('4234', '20415723', '1336527560', None, None, None, None, None, '2573411') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 419 time used for this insertion : 0.03365969657897949 save_final save missing photos in datou_result : time spend for datou_step_exec : 34.711681604385376 time spend to save output : 0.039299726486206055 total time spend for step 5 : 34.75098133087158 step6:thcl Mon Feb 10 21:21:40 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! we are using the classfication for only one thcl 3237 time to import caffe and check if the image exist : 0.006127595901489258 time to convert the images to numpy array : 0.03144645690917969 time to import caffe and check if the image exist : 0.012383699417114258 time to convert the images to numpy array : 0.03151249885559082 time to import caffe and check if the image exist : 0.01680302619934082 time to convert the images to numpy array : 0.03565192222595215 time to import caffe and check if the image exist : 0.01177978515625 time to convert the images to numpy array : 0.039368629455566406 time to import caffe and check if the image exist : 0.018409252166748047 time to convert the images to numpy array : 0.03639411926269531 time to import caffe and check if the image exist : 0.008867025375366211 time to convert the images to numpy array : 0.04798603057861328 time to import caffe and check if the image exist : 0.01090550422668457 time to convert the images to numpy array : 0.04660344123840332 time to import caffe and check if the image exist : 0.016576766967773438 time to convert the images to numpy array : 0.042252302169799805 time to import caffe and check if the image exist : 0.014488935470581055 time to convert the images to numpy array : 0.04450798034667969 time to import caffe and check if the image exist : 0.021979093551635742 time to convert the images to numpy array : 0.03768658638000488 total time to convert the images to numpy array : 0.3531360626220703 list photo_ids error: [] list photo_ids correct : [1336590338, 1336590339, 1336590340, 1336590341, 1336590342, 1336590343, 1336590344, 1336590345, 1336590346, 1336590347, 1336590348, 1336590349, 1336590350, 1336590351, 1336590367, 1336590368, 1336590369, 1336590370, 1336590371, 1336590372, 1336590373, 1336590374, 1336590375, 1336590376, 1336590377, 1336590378, 1336590379, 1336590380, 1336590440, 1336590442, 1336590443, 1336590444, 1336590445, 1336590446, 1336590447, 1336590448, 1336590455, 1336590459, 1336590463, 1336590467, 1336590471, 1336590475, 1336590532, 1336590536, 1336590539, 1336590543, 1336590547, 1336590551, 1336590555, 1336590559, 1336590563, 1336590567, 1336590571, 1336590575, 1336590579, 1336590479, 1336590483, 1336590486, 1336590489, 1336590493, 1336590497, 1336590501, 1336590505, 1336590509, 1336590512, 1336590516, 1336590520, 1336590524, 1336590528, 1336590409, 1336590410, 1336590411, 1336590412, 1336590413, 1336590414, 1336590415, 1336590418, 1336590419, 1336590420, 1336590421, 1336590422, 1336590423, 1336590424, 1336590395, 1336590396, 1336590397, 1336590398, 1336590399, 1336590400, 1336590401, 1336590402, 1336590403, 1336590404, 1336590405, 1336590406, 1336590407, 1336590408, 1336590352, 1336590353, 1336590354, 1336590356, 1336590357, 1336590358, 1336590359, 1336590360, 1336590361, 1336590362, 1336590363, 1336590364, 1336590365, 1336590366, 1336590381, 1336590382, 1336590383, 1336590384, 1336590385, 1336590386, 1336590387, 1336590388, 1336590389, 1336590390, 1336590391, 1336590392, 1336590393, 1336590394, 1336590425, 1336590426, 1336590427, 1336590428, 1336590429, 1336590430, 1336590431, 1336590432, 1336590433, 1336590434, 1336590435, 1336590436, 1336590437, 1336590438] number of photos to traite : 139 try to delete the photos incorrect in DB tagging for thcl : 3237 To do loadFromThcl(), then load ParamDescType : thcl3237 thcls : [{'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'}] thcl {'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'} Update svm_hashtag_type_desc : 5561 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) To loadFromThcl() : net_5561 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 8760 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_rubbia_refus_2500 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_rubbia_refus_2500 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_rubbia_refus_2500 /data/models_weight/learn_rubbia_refus_2500/caffemodel size_local : 94358479 size in s3 : 94358479 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:16 caffemodel already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt size_local : 32544 size in s3 : 32544 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:15 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:09:52 mean.npy already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/synset_words.txt size_local : 334 size in s3 : 334 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:10:06 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt caffemodel_filename : /data/models_weight/learn_rubbia_refus_2500/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 8541 max_wait_temp : 1 max_wait : 0 dict_keys(['pool5', 'prob']) time used to do the prepocess of the images : 0.8291387557983398 time used to do the prediction : 0.6170210838317871 save descriptor for thcl : 3237 time to traite the descriptors : 0.8859546184539795 Catched exception ! Connect or reconnect ! storage_type for insertDescriptorsMulti : 3 To insert : 1336590338 To insert : 1336590339 To insert : 1336590340 To insert : 1336590341 To insert : 1336590342 To insert : 1336590343 To insert : 1336590344 To insert : 1336590345 To insert : 1336590346 To insert : 1336590347 To insert : 1336590348 To insert : 1336590349 To insert : 1336590350 To insert : 1336590351 To insert : 1336590367 To insert : 1336590368 To insert : 1336590369 To insert : 1336590370 To insert : 1336590371 To insert : 1336590372 To insert : 1336590373 To insert : 1336590374 To insert : 1336590375 To insert : 1336590376 To insert : 1336590377 To insert : 1336590378 To insert : 1336590379 To insert : 1336590380 To insert : 1336590440 To insert : 1336590442 To insert : 1336590443 To insert : 1336590444 To insert : 1336590445 To insert : 1336590446 To insert : 1336590447 To insert : 1336590448 To insert : 1336590455 To insert : 1336590459 To insert : 1336590463 To insert : 1336590467 To insert : 1336590471 To insert : 1336590475 To insert : 1336590532 To insert : 1336590536 To insert : 1336590539 To insert : 1336590543 To insert : 1336590547 To insert : 1336590551 To insert : 1336590555 To insert : 1336590559 To insert : 1336590563 To insert : 1336590567 To insert : 1336590571 To insert : 1336590575 To insert : 1336590579 To insert : 1336590479 To insert : 1336590483 To insert : 1336590486 To insert : 1336590489 To insert : 1336590493 To insert : 1336590497 To insert : 1336590501 To insert : 1336590505 To insert : 1336590509 To insert : 1336590512 To insert : 1336590516 To insert : 1336590520 To insert : 1336590524 To insert : 1336590528 To insert : 1336590409 To insert : 1336590410 To insert : 1336590411 To insert : 1336590412 To insert : 1336590413 To insert : 1336590414 To insert : 1336590415 To insert : 1336590418 To insert : 1336590419 To insert : 1336590420 To insert : 1336590421 To insert : 1336590422 To insert : 1336590423 To insert : 1336590424 To insert : 1336590395 To insert : 1336590396 To insert : 1336590397 To insert : 1336590398 To insert : 1336590399 To insert : 1336590400 To insert : 1336590401 To insert : 1336590402 To insert : 1336590403 To insert : 1336590404 To insert : 1336590405 To insert : 1336590406 To insert : 1336590407 To insert : 1336590408 To insert : 1336590352 To insert : 1336590353 To insert : 1336590354 To insert : 1336590356 To insert : 1336590357 To insert : 1336590358 To insert : 1336590359 To insert : 1336590360 To insert : 1336590361 To insert : 1336590362 To insert : 1336590363 To insert : 1336590364 To insert : 1336590365 To insert : 1336590366 To insert : 1336590381 To insert : 1336590382 To insert : 1336590383 To insert : 1336590384 To insert : 1336590385 To insert : 1336590386 To insert : 1336590387 To insert : 1336590388 To insert : 1336590389 To insert : 1336590390 To insert : 1336590391 To insert : 1336590392 To insert : 1336590393 To insert : 1336590394 To insert : 1336590425 To insert : 1336590426 To insert : 1336590427 To insert : 1336590428 To insert : 1336590429 To insert : 1336590430 To insert : 1336590431 To insert : 1336590432 To insert : 1336590433 To insert : 1336590434 To insert : 1336590435 To insert : 1336590436 To insert : 1336590437 To insert : 1336590438 time to insert the descriptors : 22.763596773147583 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos SAVE THCL : begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 205 time used for this insertion : 0.020632028579711914 save missing photos in datou_result : time spend for datou_step_exec : 29.425074577331543 time spend to save output : 0.06713008880615234 total time spend for step 6 : 29.492204666137695 step7:ventilate_hashtags_in_portfolio Mon Feb 10 21:22:10 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 20415723 get user id for portfolio 20415723 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20415723 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('metal','autre','papier','pehd','pet_fonce','PET_clair','Papier','mal_croppe','Tetrapak','carton','pet_clair','refus','PET_fonce','Film_plastique','PEHD','Carton','environnement','flou')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20415723 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('metal','autre','papier','pehd','pet_fonce','PET_clair','Papier','mal_croppe','Tetrapak','carton','pet_clair','refus','PET_fonce','Film_plastique','PEHD','Carton','environnement','flou')) AND mptpi.`min_score`=0.6 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20415723 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('metal','autre','papier','pehd','pet_fonce','PET_clair','Papier','mal_croppe','Tetrapak','carton','pet_clair','refus','PET_fonce','Film_plastique','PEHD','Carton','environnement','flou')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20415723 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('metal','autre','papier','pehd','pet_fonce','PET_clair','Papier','mal_croppe','Tetrapak','carton','pet_clair','refus','PET_fonce','Film_plastique','PEHD','Carton','environnement','flou')) AND mptpi.`min_score`=0.6 To do lien utilise dans velours : https://www.fotonower.com/velours/20421363,20421364,20421369,20421377,20421375,20421373,20421370,20421371,20421378,20421374,20421376,20421379,20421380?tags=metal,autre,papier,pehd,pet_fonce,pet_clair,mal_croppe,tetrapak,carton,refus,film_plastique,environnement,flou&datou_id_consolidate=4235&port_consolidate=20415723 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1336527589, 1336527560] Looping around the photos to save general results len do output : 1 /20415723. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2573411') ('4234', '20415723', '1336527589', None, None, None, None, None, '2573411') ('4234', None, None, None, None, None, None, None, '2573411') ('4234', '20415723', '1336527560', None, None, None, None, None, '2573411') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.011620521545410156 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.5228869915008545 time spend to save output : 0.011840343475341797 total time spend for step 7 : 2.5347273349761963 step8:final Mon Feb 10 21:22:12 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1336527589: ('0.2176467191961314',), 1336527560: ('0.2176467191961314',)} new output for save of step final : {1336527589: ('0.2176467191961314',), 1336527560: ('0.2176467191961314',)} [1336527589, 1336527560] Looping around the photos to save general results len do output : 2 /1336527589.Didn't retrieve data . /1336527560.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2573411') ('4234', '20415723', '1336527589', None, None, None, None, None, '2573411') ('4234', None, None, None, None, None, None, None, '2573411') ('4234', '20415723', '1336527560', None, None, None, None, None, '2573411') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.011384963989257812 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.1215660572052002 time spend to save output : 0.011597394943237305 total time spend for step 8 : 0.1331634521484375 step9:velours_tree Mon Feb 10 21:22:12 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure list_portfolios : 20421363,20421364,20421369,20421377,20421375,20421373,20421370,20421371,20421378,20421374,20421376,20421379,20421380 photo desc type : 5561 - Retrieving photos to tag... query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20421363 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20421364 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20421369 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20421377 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20421375 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20421373 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20421370 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20421371 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20421378 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20421374 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20421376 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20421379 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20421380 ORDER BY ph.size desc - Loading descriptors... Size : 2048 len(descriptors) : 139 Compute structured hierarchical clustering... ward : AgglomerativeClustering(n_clusters=6) ward.labels_ : [0 5 0 0 5 1 3 5 0 1 1 5 0 0 0 0 0 1 1 4 0 0 0 0 4 0 2 0 0 0 0 4 4 0 4 5 3 0 0 5 0 0 5 1 1 3 5 1 0 0 4 0 1 5 0 5 0 0 0 0 1 1 4 0 0 0 0 0 2 0 0 0 4 0 5 3 0 4 2 0 5 0 0 5 0 3 0 0 0 2 1 0 5 5 5 3 4 0 4 1 5 0 0 4 1 5 2 4 5 4 0 0 0 1 0 4 0 4 4 5 4 4 0 4 5 2 1 0 0 0 1 1 0 0 4 4 5 0 1] Elapsed time: 0.02156996726989746 graph_id used : 77732 - Beta version, working pretty good on 11-5-16 ! https://www.fotonower.com/velours/20421363,20421364,20421369,20421377,20421375,20421373,20421370,20421371,20421378,20421374,20421376,20421379,20421380?tags=metal,autre,papier,pehd,pet_fonce,pet_clair,mal_croppe,tetrapak,carton,refus,film_plastique,environnement,flou&datou_id_consolidate=4235&port_consolidate=20415723&tree_id=77732 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : velours_tree we use saveGeneral [1336527589, 1336527560] Looping around the photos to save general results len do output : 1 /20415723Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2573411') ('4234', '20415723', '1336527589', None, None, None, None, None, '2573411') ('4234', None, None, None, None, None, None, None, '2573411') ('4234', '20415723', '1336527560', None, None, None, None, None, '2573411') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.012756586074829102 save_final save missing photos in datou_result : time spend for datou_step_exec : 9.045703172683716 time spend to save output : 0.012980937957763672 total time spend for step 9 : 9.05868411064148 step10:send_mail_cod Mon Feb 10 21:22:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Qualipapia_P20415723_10-02-2025_21_22_21.pdf 20421345 change filename to text .imagette204213451739218941 20421346 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204213461739218942 20421351 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204213511739218943 20421359 imagette204213591739218943 20421357 imagette204213571739218943 20421355 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204213551739218943 20421352 imagette204213521739218944 20421353 imagette204213531739218944 20421360 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204213601739218944 20421356 imagette204213561739218945 20421358 imagette204213581739218945 20421362 imagette204213621739218945 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20415723 and hashtag_type = 4230 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20421363,20421364,20421369,20421377,20421375,20421373,20421370,20421371,20421378,20421374,20421376,20421379,20421380?tags=metal,autre,papier,pehd,pet_fonce,pet_clair,mal_croppe,tetrapak,carton,refus,film_plastique,environnement,flou&datou_id_consolidate=4235&port_consolidate=20415723&tree_id=77732 your option no_mail is active, we will not send the real mail to your client args[1336527589] : ((1336527589, -7.736193306868757, 492609224), (1336527589, -0.337068059652412, 496442774), '0.2176467191961314') apple ((1336527589, -7.736193306868757, 492609224), (1336527589, -0.337068059652412, 496442774), '0.2176467191961314') We are sending mail with results at cod@fotonower.com args[1336527560] : ((1336527560, -7.738081110914254, 492609224), (1336527560, -0.3366222599479216, 496442774), '0.2176467191961314') apple ((1336527560, -7.738081110914254, 492609224), (1336527560, -0.3366222599479216, 496442774), '0.2176467191961314') We are sending mail with results at cod@fotonower.com refus_total : 0.2176467191961314 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20415723 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20415723_10-02-2025_21_22_21.pdf results_Qualipapia_P20415723_10-02-2025_21_22_21.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20415723_10-02-2025_21_22_21.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4234','20415723','results_Qualipapia_P20415723_10-02-2025_21_22_21.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20415723_10-02-2025_21_22_21.pdf','pdf','','0.14','0.2176467191961314') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1336527589, 1336527560] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2573411') ('4234', '20415723', '1336527589', None, None, None, None, None, '2573411') ('4234', None, None, None, None, None, None, None, '2573411') ('4234', '20415723', '1336527560', None, None, None, None, None, '2573411') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 2 time used for this insertion : 0.012143135070800781 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.077513933181763 time spend to save output : 0.012371063232421875 total time spend for step 10 : 4.089884996414185 step11:split_time_score Mon Feb 10 21:22:26 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 3442, 'mtr_user_id': 31, 'name': 'classifieur_2camions_valcor_021122_v1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'deux_camions,camion_droite,camion_gauche,pas_de_camion', 'svm_portfolios_learning': '7659379,7659034,7657685,7657114', 'photo_hashtag_type': 4458, 'photo_desc_type': 5723, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107760533,2107760534,2107760535,2107760536'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('13', 2),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 10022025 20415723 Nombre de photos uploadées : 2 / 23040 (0%) 10022025 20415723 Nombre de photos taguées (types de déchets): 2 / 2 (100%) 10022025 20415723 Nombre de photos taguées (volume) : 0 / 2 (0%) elapsed_time : load_data_split_time_score 3.0994415283203125e-06 elapsed_time : order_list_meta_photo_and_scores 1.0967254638671875e-05 ?? elapsed_time : fill_and_build_computed_from_old_data 0.00025391578674316406 elapsed_time : insert_dashboard_record_day_entry 0.02277207374572754 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20402595 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20402596 order by id desc limit 1 Qualite : 0.22689427111802013 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20402597_10-02-2025_12_47_33.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20402597 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20402597 AND mptpi.`type`=4230 To do Qualite : 0.18707302247293373 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20404181_10-02-2025_14_33_53.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20404181 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20404181 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20404182 order by id desc limit 1 Qualite : 0.24256398886930103 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20404183_10-02-2025_14_17_56.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20404183 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20404183 AND mptpi.`type`=4230 To do Qualite : 0.24593212128082215 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20404184_10-02-2025_14_15_56.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20404184 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20404184 AND mptpi.`type`=4230 To do Qualite : 0.1931657574185144 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20406214_10-02-2025_15_17_29.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20406214 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20406214 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20407528 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20407529 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20407530 order by id desc limit 1 Qualite : 0.19076758619613052 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20407531_10-02-2025_15_48_22.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20407531 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20407531 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20407532 order by id desc limit 1 Qualite : 0.19477289041868498 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20408736_10-02-2025_16_17_51.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20408736 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20408736 AND mptpi.`type`=4230 To do Qualite : 0.19823456819653626 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20408737_10-02-2025_16_32_46.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20408737 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20408737 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20409931 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20409932 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20409933 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20409934 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20409935 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20409936 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20409937 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20409938 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20409939 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20409952 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20409965 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20409966 order by id desc limit 1 Qualite : 0.23387791603237645 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20409967_10-02-2025_16_47_36.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20409967 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20409967 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20409968 order by id desc limit 1 Qualite : 0.2176467191961314 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20415723_10-02-2025_21_22_21.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20415723 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20415723 AND mptpi.`type`=4230 To do Qualite : 0.19383890763343226 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20415724_10-02-2025_20_48_31.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20415724 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20415724 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20411225 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20411226 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20411227 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20411228 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20411229 order by id desc limit 1 Qualite : 0.12786421795977349 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20411230_10-02-2025_17_32_34.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20411230 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20411230 AND mptpi.`type`=4230 To do Qualite : 0.1820284055376528 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20411231_10-02-2025_17_17_32.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20411231 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20411231 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20414518 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20414519 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20414520 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20414521 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20412588 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20412589 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20412590 order by id desc limit 1 Qualite : 0.17137703988881423 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20412591_10-02-2025_17_48_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20412591 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20412591 AND mptpi.`type`=4230 To do Qualite : 0.24645736969852525 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20412592_10-02-2025_17_35_23.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20412592 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20412592 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20413215 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20413216 order by id desc limit 1 Qualite : 0.16731784552960752 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20413217_10-02-2025_18_34_52.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20413217 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20413217 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20413218 order by id desc limit 1 Qualite : 0.17117586894921286 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20413219_10-02-2025_18_19_26.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20413219 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20413219 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20414522 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20414523 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20414524 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20414525 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20414526 order by id desc limit 1 Qualite : 0.17529224132018398 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20414527_10-02-2025_18_52_32.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20414527 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20414527 AND mptpi.`type`=4230 To do Qualite : 0.26397665981018675 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20415725_10-02-2025_19_32_27.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20415725 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20415725 AND mptpi.`type`=4230 To do Qualite : 0.28135115983410103 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20415726_10-02-2025_19_17_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20415726 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20415726 AND mptpi.`type`=4230 To do Qualite : 0.19575669186025568 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20415727_10-02-2025_20_33_55.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20415727 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20415727 AND mptpi.`type`=4230 To do Qualite : 0.17243178524849828 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20416894_10-02-2025_20_22_52.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20416894 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20416894 AND mptpi.`type`=4230 To do Qualite : 0.20290222051769696 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20416895_10-02-2025_19_57_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20416895 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20416895 AND mptpi.`type`=4230 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'10022025': {'nb_upload': 2, 'nb_taggue_class': 2, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1336527589, 1336527560] Looping around the photos to save general results len do output : 1 /20415723Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2573411') ('4234', '20415723', '1336527589', None, None, None, None, None, '2573411') ('4234', None, None, None, None, None, None, None, '2573411') ('4234', '20415723', '1336527560', None, None, None, None, None, '2573411') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.012788534164428711 save_final save missing photos in datou_result : time spend for datou_step_exec : 34.66195750236511 time spend to save output : 0.012919902801513672 total time spend for step 11 : 34.674877405166626 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 1 len(input) + len(total_photo_id_missing) : 2 set_done_treatment 33.41user 23.58system 2:36.19elapsed 36%CPU (0avgtext+0avgdata 2519700maxresident)k 826488inputs+11464outputs (2253major+2162924minor)pagefaults 0swaps