python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 4234 -l 5 ' -s datou_current_4234 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3821838 load datou : 4234 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load THCL from format json or kwargs add thcl : 3237 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig add pdt : 5561 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4234, datou_cur_ids : ['2569149'] with mtr_portfolio_ids : ['20363428'] and first list_photo_ids : [] new path : /proc/3821838/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, brightness, blur_detection, rle_unique_nms_with_priority, crop_condition, thcl, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , BFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 2 ; length of list_pids : 2 ; length of list_args : 2 time to download the photos : 0.3151085376739502 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Sun Feb 9 16:20:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10553 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-09 16:20:31.925194: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-09 16:20:31.951155: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-09 16:20:31.953214: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f3468000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-09 16:20:31.953245: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-09 16:20:31.957313: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-09 16:20:32.192945: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2ddf6330 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-09 16:20:32.192994: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-09 16:20:32.194369: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-09 16:20:32.195017: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-09 16:20:32.199481: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-09 16:20:32.204686: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-09 16:20:32.205090: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-09 16:20:32.208667: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-09 16:20:32.211300: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-09 16:20:32.222055: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-09 16:20:32.223876: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-09 16:20:32.223965: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-09 16:20:32.224904: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-09 16:20:32.224923: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-09 16:20:32.224935: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-09 16:20:32.226571: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9777 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-09 16:20:32.519843: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-09 16:20:32.520011: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-09 16:20:32.520046: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-09 16:20:32.520076: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-09 16:20:32.520107: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-09 16:20:32.520135: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-09 16:20:32.520163: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-09 16:20:32.520192: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-09 16:20:32.521536: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-09 16:20:32.522881: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-09 16:20:32.522951: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-09 16:20:32.522970: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-09 16:20:32.522984: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-09 16:20:32.522998: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-09 16:20:32.523012: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-09 16:20:32.523026: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-09 16:20:32.523040: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-09 16:20:32.524289: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-09 16:20:32.524325: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-09 16:20:32.524333: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-09 16:20:32.524341: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-09 16:20:32.525649: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9777 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-09 16:20:39.847025: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-09 16:20:40.031430: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 2 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -119.33281 max: 148.18203 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 0.000102996826171875 nb_pixel_total : 370 time to create 1 rle with old method : 0.0008566379547119141 length of segment : 32 time for calcul the mask position with numpy : 5.936622619628906e-05 nb_pixel_total : 237 time to create 1 rle with old method : 0.0005609989166259766 length of segment : 43 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 129 time to create 1 rle with old method : 0.00027680397033691406 length of segment : 27 time for calcul the mask position with numpy : 6.580352783203125e-05 nb_pixel_total : 836 time to create 1 rle with old method : 0.001867055892944336 length of segment : 35 time for calcul the mask position with numpy : 5.555152893066406e-05 nb_pixel_total : 828 time to create 1 rle with old method : 0.0014119148254394531 length of segment : 35 time for calcul the mask position with numpy : 6.151199340820312e-05 nb_pixel_total : 874 time to create 1 rle with old method : 0.0014424324035644531 length of segment : 31 time for calcul the mask position with numpy : 0.00019025802612304688 nb_pixel_total : 3039 time to create 1 rle with old method : 0.005248546600341797 length of segment : 86 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -121.05938 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 8.416175842285156e-05 nb_pixel_total : 2422 time to create 1 rle with old method : 0.0031511783599853516 length of segment : 52 time for calcul the mask position with numpy : 3.4332275390625e-05 nb_pixel_total : 13 time to create 1 rle with old method : 5.221366882324219e-05 length of segment : 10 time for calcul the mask position with numpy : 5.2928924560546875e-05 nb_pixel_total : 356 time to create 1 rle with old method : 0.0006117820739746094 length of segment : 41 Processing 1 images image shape: (400, 400, 3) min: 1.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -104.88750 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 0.0001304149627685547 nb_pixel_total : 6821 time to create 1 rle with old method : 0.00902247428894043 length of segment : 110 length of segment : 0 time for calcul the mask position with numpy : 7.843971252441406e-05 nb_pixel_total : 1773 time to create 1 rle with old method : 0.002794027328491211 length of segment : 27 time for calcul the mask position with numpy : 0.00014209747314453125 nb_pixel_total : 2351 time to create 1 rle with old method : 0.003576517105102539 length of segment : 125 time for calcul the mask position with numpy : 0.00010609626770019531 nb_pixel_total : 1137 time to create 1 rle with old method : 0.0019185543060302734 length of segment : 86 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.09063 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 6.771087646484375e-05 nb_pixel_total : 438 time to create 1 rle with old method : 0.001094818115234375 length of segment : 26 time for calcul the mask position with numpy : 0.0001499652862548828 nb_pixel_total : 9563 time to create 1 rle with old method : 0.012607097625732422 length of segment : 114 time for calcul the mask position with numpy : 0.00037479400634765625 nb_pixel_total : 23977 time to create 1 rle with old method : 0.02655816078186035 length of segment : 401 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 248.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 129.39297 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 4.673004150390625e-05 nb_pixel_total : 129 time to create 1 rle with old method : 0.0002682209014892578 length of segment : 12 time for calcul the mask position with numpy : 4.172325134277344e-05 nb_pixel_total : 643 time to create 1 rle with old method : 0.0008428096771240234 length of segment : 48 time for calcul the mask position with numpy : 3.0517578125e-05 nb_pixel_total : 140 time to create 1 rle with old method : 0.0002391338348388672 length of segment : 12 time for calcul the mask position with numpy : 3.170967102050781e-05 nb_pixel_total : 173 time to create 1 rle with old method : 0.0002894401550292969 length of segment : 20 Processing 1 images image shape: (400, 400, 3) min: 17.00000 max: 186.00000 molded_images shape: (1, 640, 640, 3) min: -88.81719 max: 62.82500 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -118.83672 max: 149.01797 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 11 time for calcul the mask position with numpy : 5.078315734863281e-05 nb_pixel_total : 394 time to create 1 rle with old method : 0.0005676746368408203 length of segment : 20 time for calcul the mask position with numpy : 3.1948089599609375e-05 nb_pixel_total : 140 time to create 1 rle with old method : 0.00022077560424804688 length of segment : 13 time for calcul the mask position with numpy : 4.00543212890625e-05 nb_pixel_total : 846 time to create 1 rle with old method : 0.001220703125 length of segment : 26 time for calcul the mask position with numpy : 3.24249267578125e-05 nb_pixel_total : 290 time to create 1 rle with old method : 0.00043320655822753906 length of segment : 18 time for calcul the mask position with numpy : 3.7670135498046875e-05 nb_pixel_total : 654 time to create 1 rle with old method : 0.0009438991546630859 length of segment : 19 time for calcul the mask position with numpy : 3.123283386230469e-05 nb_pixel_total : 291 time to create 1 rle with old method : 0.0004444122314453125 length of segment : 13 time for calcul the mask position with numpy : 2.9325485229492188e-05 nb_pixel_total : 88 time to create 1 rle with old method : 0.00019311904907226562 length of segment : 8 time for calcul the mask position with numpy : 3.886222839355469e-05 nb_pixel_total : 837 time to create 1 rle with old method : 0.0012066364288330078 length of segment : 26 time for calcul the mask position with numpy : 2.956390380859375e-05 nb_pixel_total : 121 time to create 1 rle with old method : 0.0002014636993408203 length of segment : 11 time for calcul the mask position with numpy : 3.719329833984375e-05 nb_pixel_total : 695 time to create 1 rle with old method : 0.0009629726409912109 length of segment : 29 time for calcul the mask position with numpy : 4.3392181396484375e-05 nb_pixel_total : 851 time to create 1 rle with old method : 0.0012021064758300781 length of segment : 36 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -110.16875 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 13 time for calcul the mask position with numpy : 5.1021575927734375e-05 nb_pixel_total : 168 time to create 1 rle with old method : 0.00031757354736328125 length of segment : 18 time for calcul the mask position with numpy : 5.459785461425781e-05 nb_pixel_total : 1803 time to create 1 rle with old method : 0.0022640228271484375 length of segment : 38 time for calcul the mask position with numpy : 6.0558319091796875e-05 nb_pixel_total : 1061 time to create 1 rle with old method : 0.0014972686767578125 length of segment : 49 time for calcul the mask position with numpy : 5.841255187988281e-05 nb_pixel_total : 1313 time to create 1 rle with old method : 0.0017864704132080078 length of segment : 51 time for calcul the mask position with numpy : 4.029273986816406e-05 nb_pixel_total : 507 time to create 1 rle with old method : 0.0007307529449462891 length of segment : 30 time for calcul the mask position with numpy : 3.552436828613281e-05 nb_pixel_total : 457 time to create 1 rle with old method : 0.0006809234619140625 length of segment : 20 length of segment : 0 time for calcul the mask position with numpy : 2.765655517578125e-05 nb_pixel_total : 36 time to create 1 rle with old method : 8.058547973632812e-05 length of segment : 6 time for calcul the mask position with numpy : 3.1948089599609375e-05 nb_pixel_total : 417 time to create 1 rle with old method : 0.0005495548248291016 length of segment : 29 time for calcul the mask position with numpy : 2.9325485229492188e-05 nb_pixel_total : 114 time to create 1 rle with old method : 0.0001842975616455078 length of segment : 18 time for calcul the mask position with numpy : 2.956390380859375e-05 nb_pixel_total : 162 time to create 1 rle with old method : 0.0002503395080566406 length of segment : 15 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 885 time to create 1 rle with old method : 0.0011255741119384766 length of segment : 53 time for calcul the mask position with numpy : 5.2928924560546875e-05 nb_pixel_total : 1800 time to create 1 rle with old method : 0.0022292137145996094 length of segment : 46 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.64531 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 8.511543273925781e-05 nb_pixel_total : 1692 time to create 1 rle with old method : 0.002044200897216797 length of segment : 131 time for calcul the mask position with numpy : 3.4809112548828125e-05 nb_pixel_total : 110 time to create 1 rle with old method : 0.00017786026000976562 length of segment : 35 time for calcul the mask position with numpy : 0.00036644935607910156 nb_pixel_total : 3594 time to create 1 rle with old method : 0.0052683353424072266 length of segment : 215 time for calcul the mask position with numpy : 6.723403930664062e-05 nb_pixel_total : 2321 time to create 1 rle with old method : 0.002831697463989258 length of segment : 55 time for calcul the mask position with numpy : 0.0008356571197509766 nb_pixel_total : 68540 time to create 1 rle with old method : 0.07710742950439453 length of segment : 389 time for calcul the mask position with numpy : 3.814697265625e-05 nb_pixel_total : 47 time to create 1 rle with old method : 0.000102996826171875 length of segment : 7 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 253.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 124.80312 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 4.076957702636719e-05 nb_pixel_total : 405 time to create 1 rle with old method : 0.0005652904510498047 length of segment : 38 time for calcul the mask position with numpy : 0.00010538101196289062 nb_pixel_total : 4460 time to create 1 rle with old method : 0.005179166793823242 length of segment : 184 time for calcul the mask position with numpy : 5.221366882324219e-05 nb_pixel_total : 660 time to create 1 rle with old method : 0.0008764266967773438 length of segment : 61 time for calcul the mask position with numpy : 0.0001804828643798828 nb_pixel_total : 4967 time to create 1 rle with old method : 0.006327629089355469 length of segment : 278 time for calcul the mask position with numpy : 4.76837158203125e-05 nb_pixel_total : 554 time to create 1 rle with old method : 0.0007979869842529297 length of segment : 45 time for calcul the mask position with numpy : 6.604194641113281e-05 nb_pixel_total : 925 time to create 1 rle with old method : 0.0013797283172607422 length of segment : 99 time for calcul the mask position with numpy : 5.5789947509765625e-05 nb_pixel_total : 704 time to create 1 rle with old method : 0.000997304916381836 length of segment : 62 Processing 1 images image shape: (280, 400, 3) min: 9.00000 max: 184.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 58.20156 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.001027822494506836 nb_pixel_total : 106420 time to create 1 rle with old method : 0.13518810272216797 length of segment : 281 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 148.22500 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 13 time for calcul the mask position with numpy : 4.6253204345703125e-05 nb_pixel_total : 378 time to create 1 rle with old method : 0.0005958080291748047 length of segment : 43 time for calcul the mask position with numpy : 3.600120544433594e-05 nb_pixel_total : 218 time to create 1 rle with old method : 0.00034332275390625 length of segment : 19 time for calcul the mask position with numpy : 3.2901763916015625e-05 nb_pixel_total : 276 time to create 1 rle with old method : 0.00042891502380371094 length of segment : 17 time for calcul the mask position with numpy : 3.7670135498046875e-05 nb_pixel_total : 499 time to create 1 rle with old method : 0.0006811618804931641 length of segment : 32 time for calcul the mask position with numpy : 4.8160552978515625e-05 nb_pixel_total : 820 time to create 1 rle with old method : 0.0010895729064941406 length of segment : 51 time for calcul the mask position with numpy : 3.1948089599609375e-05 nb_pixel_total : 214 time to create 1 rle with old method : 0.0003352165222167969 length of segment : 17 time for calcul the mask position with numpy : 3.600120544433594e-05 nb_pixel_total : 469 time to create 1 rle with old method : 0.0006844997406005859 length of segment : 32 time for calcul the mask position with numpy : 4.482269287109375e-05 nb_pixel_total : 937 time to create 1 rle with old method : 0.0012483596801757812 length of segment : 36 time for calcul the mask position with numpy : 3.552436828613281e-05 nb_pixel_total : 306 time to create 1 rle with old method : 0.0004405975341796875 length of segment : 27 time for calcul the mask position with numpy : 3.695487976074219e-05 nb_pixel_total : 580 time to create 1 rle with old method : 0.0008304119110107422 length of segment : 25 time for calcul the mask position with numpy : 5.435943603515625e-05 nb_pixel_total : 981 time to create 1 rle with old method : 0.0013041496276855469 length of segment : 65 time for calcul the mask position with numpy : 3.719329833984375e-05 nb_pixel_total : 357 time to create 1 rle with old method : 0.0005061626434326172 length of segment : 18 time for calcul the mask position with numpy : 5.340576171875e-05 nb_pixel_total : 853 time to create 1 rle with old method : 0.001165151596069336 length of segment : 33 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.81875 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 14 time for calcul the mask position with numpy : 5.2928924560546875e-05 nb_pixel_total : 584 time to create 1 rle with old method : 0.000736236572265625 length of segment : 45 time for calcul the mask position with numpy : 3.695487976074219e-05 nb_pixel_total : 222 time to create 1 rle with old method : 0.0003044605255126953 length of segment : 21 time for calcul the mask position with numpy : 3.266334533691406e-05 nb_pixel_total : 102 time to create 1 rle with old method : 0.0001742839813232422 length of segment : 14 time for calcul the mask position with numpy : 3.981590270996094e-05 nb_pixel_total : 78 time to create 1 rle with old method : 0.0001876354217529297 length of segment : 24 time for calcul the mask position with numpy : 3.933906555175781e-05 nb_pixel_total : 305 time to create 1 rle with old method : 0.0005023479461669922 length of segment : 29 time for calcul the mask position with numpy : 5.054473876953125e-05 nb_pixel_total : 742 time to create 1 rle with old method : 0.0010309219360351562 length of segment : 39 time for calcul the mask position with numpy : 3.6716461181640625e-05 nb_pixel_total : 164 time to create 1 rle with old method : 0.00024366378784179688 length of segment : 20 time for calcul the mask position with numpy : 5.4836273193359375e-05 nb_pixel_total : 1244 time to create 1 rle with old method : 0.0016393661499023438 length of segment : 52 time for calcul the mask position with numpy : 4.100799560546875e-05 nb_pixel_total : 277 time to create 1 rle with old method : 0.00043654441833496094 length of segment : 17 time for calcul the mask position with numpy : 4.935264587402344e-05 nb_pixel_total : 370 time to create 1 rle with old method : 0.0005986690521240234 length of segment : 40 time for calcul the mask position with numpy : 3.910064697265625e-05 nb_pixel_total : 174 time to create 1 rle with old method : 0.00029754638671875 length of segment : 17 time for calcul the mask position with numpy : 2.956390380859375e-05 nb_pixel_total : 109 time to create 1 rle with old method : 0.00017571449279785156 length of segment : 11 time for calcul the mask position with numpy : 3.600120544433594e-05 nb_pixel_total : 366 time to create 1 rle with old method : 0.0005366802215576172 length of segment : 22 time for calcul the mask position with numpy : 3.147125244140625e-05 nb_pixel_total : 220 time to create 1 rle with old method : 0.00032711029052734375 length of segment : 17 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 146.76406 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 12 time for calcul the mask position with numpy : 5.245208740234375e-05 nb_pixel_total : 1043 time to create 1 rle with old method : 0.0013363361358642578 length of segment : 51 time for calcul the mask position with numpy : 3.790855407714844e-05 nb_pixel_total : 473 time to create 1 rle with old method : 0.0005893707275390625 length of segment : 30 time for calcul the mask position with numpy : 3.457069396972656e-05 nb_pixel_total : 379 time to create 1 rle with old method : 0.000568389892578125 length of segment : 19 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 740 time to create 1 rle with old method : 0.0009737014770507812 length of segment : 56 time for calcul the mask position with numpy : 3.528594970703125e-05 nb_pixel_total : 238 time to create 1 rle with old method : 0.0003712177276611328 length of segment : 33 time for calcul the mask position with numpy : 4.267692565917969e-05 nb_pixel_total : 72 time to create 1 rle with old method : 0.00012946128845214844 length of segment : 18 time for calcul the mask position with numpy : 2.9087066650390625e-05 nb_pixel_total : 112 time to create 1 rle with old method : 0.00018596649169921875 length of segment : 10 time for calcul the mask position with numpy : 0.0005135536193847656 nb_pixel_total : 43064 time to create 1 rle with old method : 0.05022692680358887 length of segment : 261 time for calcul the mask position with numpy : 5.125999450683594e-05 nb_pixel_total : 388 time to create 1 rle with old method : 0.0005564689636230469 length of segment : 23 time for calcul the mask position with numpy : 0.00061798095703125 nb_pixel_total : 42995 time to create 1 rle with old method : 0.051737308502197266 length of segment : 232 time for calcul the mask position with numpy : 0.0004763603210449219 nb_pixel_total : 31950 time to create 1 rle with old method : 0.03867936134338379 length of segment : 264 time for calcul the mask position with numpy : 0.0004963874816894531 nb_pixel_total : 37929 time to create 1 rle with old method : 0.04333138465881348 length of segment : 225 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 141.28750 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 0.00011610984802246094 nb_pixel_total : 692 time to create 1 rle with old method : 0.001432657241821289 length of segment : 78 time for calcul the mask position with numpy : 0.0005013942718505859 nb_pixel_total : 41669 time to create 1 rle with old method : 0.046952009201049805 length of segment : 252 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -119.38359 max: 144.87734 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 15 time for calcul the mask position with numpy : 5.745887756347656e-05 nb_pixel_total : 81 time to create 1 rle with old method : 0.00014281272888183594 length of segment : 29 time for calcul the mask position with numpy : 0.00037789344787597656 nb_pixel_total : 8598 time to create 1 rle with old method : 0.010833024978637695 length of segment : 182 time for calcul the mask position with numpy : 0.0002994537353515625 nb_pixel_total : 6762 time to create 1 rle with old method : 0.008625268936157227 length of segment : 159 time for calcul the mask position with numpy : 0.00012421607971191406 nb_pixel_total : 2858 time to create 1 rle with old method : 0.003658771514892578 length of segment : 69 time for calcul the mask position with numpy : 3.838539123535156e-05 nb_pixel_total : 279 time to create 1 rle with old method : 0.0004050731658935547 length of segment : 25 time for calcul the mask position with numpy : 3.409385681152344e-05 nb_pixel_total : 310 time to create 1 rle with old method : 0.00047397613525390625 length of segment : 44 time for calcul the mask position with numpy : 4.291534423828125e-05 nb_pixel_total : 666 time to create 1 rle with old method : 0.0009868144989013672 length of segment : 37 time for calcul the mask position with numpy : 3.361701965332031e-05 nb_pixel_total : 412 time to create 1 rle with old method : 0.0005729198455810547 length of segment : 35 time for calcul the mask position with numpy : 3.170967102050781e-05 nb_pixel_total : 226 time to create 1 rle with old method : 0.00034809112548828125 length of segment : 39 time for calcul the mask position with numpy : 9.72747802734375e-05 nb_pixel_total : 2224 time to create 1 rle with old method : 0.002796173095703125 length of segment : 92 time for calcul the mask position with numpy : 3.8623809814453125e-05 nb_pixel_total : 7 time to create 1 rle with old method : 0.00011110305786132812 length of segment : 5 time for calcul the mask position with numpy : 3.123283386230469e-05 nb_pixel_total : 131 time to create 1 rle with old method : 0.00024247169494628906 length of segment : 19 time for calcul the mask position with numpy : 4.1484832763671875e-05 nb_pixel_total : 841 time to create 1 rle with old method : 0.0011949539184570312 length of segment : 37 time for calcul the mask position with numpy : 5.984306335449219e-05 nb_pixel_total : 1215 time to create 1 rle with old method : 0.0016760826110839844 length of segment : 50 time for calcul the mask position with numpy : 3.4809112548828125e-05 nb_pixel_total : 248 time to create 1 rle with old method : 0.0003733634948730469 length of segment : 36 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -121.05938 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 4.9591064453125e-05 nb_pixel_total : 506 time to create 1 rle with old method : 0.0008754730224609375 length of segment : 28 time for calcul the mask position with numpy : 3.647804260253906e-05 nb_pixel_total : 499 time to create 1 rle with old method : 0.0006508827209472656 length of segment : 33 time for calcul the mask position with numpy : 6.341934204101562e-05 nb_pixel_total : 2423 time to create 1 rle with old method : 0.0032377243041992188 length of segment : 51 time for calcul the mask position with numpy : 3.6716461181640625e-05 nb_pixel_total : 534 time to create 1 rle with old method : 0.0007269382476806641 length of segment : 30 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -110.03984 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 0.00019478797912597656 nb_pixel_total : 7367 time to create 1 rle with old method : 0.00868678092956543 length of segment : 106 time for calcul the mask position with numpy : 5.650520324707031e-05 nb_pixel_total : 827 time to create 1 rle with old method : 0.0012302398681640625 length of segment : 29 time for calcul the mask position with numpy : 6.508827209472656e-05 nb_pixel_total : 1741 time to create 1 rle with old method : 0.0025985240936279297 length of segment : 20 time for calcul the mask position with numpy : 0.00013375282287597656 nb_pixel_total : 1608 time to create 1 rle with old method : 0.002415180206298828 length of segment : 99 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.80938 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 4.649162292480469e-05 nb_pixel_total : 453 time to create 1 rle with old method : 0.0007224082946777344 length of segment : 26 time for calcul the mask position with numpy : 0.0003941059112548828 nb_pixel_total : 23484 time to create 1 rle with old method : 0.027716636657714844 length of segment : 322 time for calcul the mask position with numpy : 0.00019669532775878906 nb_pixel_total : 5931 time to create 1 rle with old method : 0.0073909759521484375 length of segment : 158 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 253.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 131.04531 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 4.6253204345703125e-05 nb_pixel_total : 131 time to create 1 rle with old method : 0.00026798248291015625 length of segment : 11 time for calcul the mask position with numpy : 0.0005812644958496094 nb_pixel_total : 45121 time to create 1 rle with old method : 0.051471710205078125 length of segment : 275 Processing 1 images image shape: (400, 400, 3) min: 13.00000 max: 202.00000 molded_images shape: (1, 640, 640, 3) min: -89.36406 max: 70.50234 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -119.95391 max: 150.48672 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 13 time for calcul the mask position with numpy : 4.00543212890625e-05 nb_pixel_total : 140 time to create 1 rle with old method : 0.0002033710479736328 length of segment : 15 time for calcul the mask position with numpy : 3.170967102050781e-05 nb_pixel_total : 314 time to create 1 rle with old method : 0.0004353523254394531 length of segment : 19 time for calcul the mask position with numpy : 3.457069396972656e-05 nb_pixel_total : 359 time to create 1 rle with old method : 0.0005390644073486328 length of segment : 19 time for calcul the mask position with numpy : 3.218650817871094e-05 nb_pixel_total : 239 time to create 1 rle with old method : 0.0003714561462402344 length of segment : 17 time for calcul the mask position with numpy : 3.790855407714844e-05 nb_pixel_total : 797 time to create 1 rle with old method : 0.0011334419250488281 length of segment : 26 time for calcul the mask position with numpy : 3.266334533691406e-05 nb_pixel_total : 284 time to create 1 rle with old method : 0.00046133995056152344 length of segment : 17 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 891 time to create 1 rle with old method : 0.0012900829315185547 length of segment : 19 time for calcul the mask position with numpy : 3.0040740966796875e-05 nb_pixel_total : 69 time to create 1 rle with old method : 0.00011682510375976562 length of segment : 20 time for calcul the mask position with numpy : 0.00010228157043457031 nb_pixel_total : 5414 time to create 1 rle with old method : 0.006975889205932617 length of segment : 86 time for calcul the mask position with numpy : 6.151199340820312e-05 nb_pixel_total : 2121 time to create 1 rle with old method : 0.0027670860290527344 length of segment : 70 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 102 time to create 1 rle with old method : 0.00016641616821289062 length of segment : 9 time for calcul the mask position with numpy : 9.5367431640625e-05 nb_pixel_total : 774 time to create 1 rle with old method : 0.0009355545043945312 length of segment : 31 time for calcul the mask position with numpy : 7.319450378417969e-05 nb_pixel_total : 856 time to create 1 rle with old method : 0.0011162757873535156 length of segment : 26 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -112.78203 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 10 time for calcul the mask position with numpy : 5.4836273193359375e-05 nb_pixel_total : 837 time to create 1 rle with old method : 0.0011837482452392578 length of segment : 29 time for calcul the mask position with numpy : 4.482269287109375e-05 nb_pixel_total : 221 time to create 1 rle with old method : 0.00038933753967285156 length of segment : 18 time for calcul the mask position with numpy : 3.981590270996094e-05 nb_pixel_total : 451 time to create 1 rle with old method : 0.0005943775177001953 length of segment : 23 time for calcul the mask position with numpy : 3.814697265625e-05 nb_pixel_total : 129 time to create 1 rle with old method : 0.0002639293670654297 length of segment : 17 time for calcul the mask position with numpy : 4.863739013671875e-05 nb_pixel_total : 864 time to create 1 rle with old method : 0.0011279582977294922 length of segment : 61 time for calcul the mask position with numpy : 7.104873657226562e-05 nb_pixel_total : 670 time to create 1 rle with old method : 0.0012848377227783203 length of segment : 32 time for calcul the mask position with numpy : 5.841255187988281e-05 nb_pixel_total : 1476 time to create 1 rle with old method : 0.0019812583923339844 length of segment : 41 time for calcul the mask position with numpy : 6.413459777832031e-05 nb_pixel_total : 1607 time to create 1 rle with old method : 0.001981019973754883 length of segment : 59 time for calcul the mask position with numpy : 7.009506225585938e-05 nb_pixel_total : 1673 time to create 1 rle with old method : 0.0022215843200683594 length of segment : 60 time for calcul the mask position with numpy : 5.817413330078125e-05 nb_pixel_total : 1319 time to create 1 rle with old method : 0.0016896724700927734 length of segment : 38 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.64531 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 8.630752563476562e-05 nb_pixel_total : 1956 time to create 1 rle with old method : 0.00238800048828125 length of segment : 127 time for calcul the mask position with numpy : 0.00021767616271972656 nb_pixel_total : 12741 time to create 1 rle with old method : 0.014907598495483398 length of segment : 175 time for calcul the mask position with numpy : 4.792213439941406e-05 nb_pixel_total : 41 time to create 1 rle with old method : 8.940696716308594e-05 length of segment : 16 time for calcul the mask position with numpy : 3.170967102050781e-05 nb_pixel_total : 46 time to create 1 rle with old method : 0.00010371208190917969 length of segment : 8 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 251.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 127.76016 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 4.792213439941406e-05 nb_pixel_total : 417 time to create 1 rle with old method : 0.0005884170532226562 length of segment : 41 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 828 time to create 1 rle with old method : 0.0010695457458496094 length of segment : 35 time for calcul the mask position with numpy : 7.319450378417969e-05 nb_pixel_total : 1163 time to create 1 rle with old method : 0.0016560554504394531 length of segment : 96 time for calcul the mask position with numpy : 4.673004150390625e-05 nb_pixel_total : 454 time to create 1 rle with old method : 0.0006465911865234375 length of segment : 30 time for calcul the mask position with numpy : 4.8160552978515625e-05 nb_pixel_total : 815 time to create 1 rle with old method : 0.001178741455078125 length of segment : 35 time for calcul the mask position with numpy : 3.790855407714844e-05 nb_pixel_total : 214 time to create 1 rle with old method : 0.000316619873046875 length of segment : 26 time for calcul the mask position with numpy : 6.246566772460938e-05 nb_pixel_total : 659 time to create 1 rle with old method : 0.0009245872497558594 length of segment : 64 time for calcul the mask position with numpy : 0.00033283233642578125 nb_pixel_total : 5220 time to create 1 rle with old method : 0.0071599483489990234 length of segment : 291 Processing 1 images image shape: (280, 400, 3) min: 3.00000 max: 184.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 57.76797 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.00099945068359375 nb_pixel_total : 106136 time to create 1 rle with old method : 0.12337899208068848 length of segment : 282 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 148.51016 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 4.267692565917969e-05 nb_pixel_total : 173 time to create 1 rle with old method : 0.0002627372741699219 length of segment : 17 time for calcul the mask position with numpy : 4.410743713378906e-05 nb_pixel_total : 446 time to create 1 rle with old method : 0.0006222724914550781 length of segment : 33 time for calcul the mask position with numpy : 4.315376281738281e-05 nb_pixel_total : 353 time to create 1 rle with old method : 0.0005469322204589844 length of segment : 18 time for calcul the mask position with numpy : 4.172325134277344e-05 nb_pixel_total : 354 time to create 1 rle with old method : 0.0005145072937011719 length of segment : 30 time for calcul the mask position with numpy : 5.555152893066406e-05 nb_pixel_total : 1048 time to create 1 rle with old method : 0.00138092041015625 length of segment : 46 time for calcul the mask position with numpy : 7.009506225585938e-05 nb_pixel_total : 1567 time to create 1 rle with old method : 0.0020055770874023438 length of segment : 57 time for calcul the mask position with numpy : 4.601478576660156e-05 nb_pixel_total : 544 time to create 1 rle with old method : 0.0007936954498291016 length of segment : 24 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.77187 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 22 time for calcul the mask position with numpy : 5.626678466796875e-05 nb_pixel_total : 194 time to create 1 rle with old method : 0.00029778480529785156 length of segment : 18 time for calcul the mask position with numpy : 3.886222839355469e-05 nb_pixel_total : 247 time to create 1 rle with old method : 0.0003814697265625 length of segment : 17 time for calcul the mask position with numpy : 4.458427429199219e-05 nb_pixel_total : 639 time to create 1 rle with old method : 0.0008857250213623047 length of segment : 43 time for calcul the mask position with numpy : 3.814697265625e-05 nb_pixel_total : 101 time to create 1 rle with old method : 0.00018072128295898438 length of segment : 14 time for calcul the mask position with numpy : 3.7670135498046875e-05 nb_pixel_total : 228 time to create 1 rle with old method : 0.0003345012664794922 length of segment : 22 time for calcul the mask position with numpy : 3.695487976074219e-05 nb_pixel_total : 272 time to create 1 rle with old method : 0.0004253387451171875 length of segment : 17 time for calcul the mask position with numpy : 3.314018249511719e-05 nb_pixel_total : 131 time to create 1 rle with old method : 0.00021767616271972656 length of segment : 15 time for calcul the mask position with numpy : 3.695487976074219e-05 nb_pixel_total : 276 time to create 1 rle with old method : 0.0004436969757080078 length of segment : 20 time for calcul the mask position with numpy : 6.127357482910156e-05 nb_pixel_total : 292 time to create 1 rle with old method : 0.0004642009735107422 length of segment : 21 time for calcul the mask position with numpy : 6.008148193359375e-05 nb_pixel_total : 243 time to create 1 rle with old method : 0.0004115104675292969 length of segment : 13 time for calcul the mask position with numpy : 3.314018249511719e-05 nb_pixel_total : 124 time to create 1 rle with old method : 0.000194549560546875 length of segment : 17 time for calcul the mask position with numpy : 3.528594970703125e-05 nb_pixel_total : 204 time to create 1 rle with old method : 0.0002903938293457031 length of segment : 22 time for calcul the mask position with numpy : 3.5762786865234375e-05 nb_pixel_total : 251 time to create 1 rle with old method : 0.0003631114959716797 length of segment : 26 time for calcul the mask position with numpy : 5.412101745605469e-05 nb_pixel_total : 288 time to create 1 rle with old method : 0.00044155120849609375 length of segment : 17 time for calcul the mask position with numpy : 4.315376281738281e-05 nb_pixel_total : 70 time to create 1 rle with old method : 0.00013518333435058594 length of segment : 11 time for calcul the mask position with numpy : 3.6716461181640625e-05 nb_pixel_total : 376 time to create 1 rle with old method : 0.0005214214324951172 length of segment : 23 time for calcul the mask position with numpy : 4.673004150390625e-05 nb_pixel_total : 609 time to create 1 rle with old method : 0.0008959770202636719 length of segment : 29 time for calcul the mask position with numpy : 3.814697265625e-05 nb_pixel_total : 59 time to create 1 rle with old method : 0.00013709068298339844 length of segment : 8 time for calcul the mask position with numpy : 4.9591064453125e-05 nb_pixel_total : 710 time to create 1 rle with old method : 0.0009691715240478516 length of segment : 29 time for calcul the mask position with numpy : 3.719329833984375e-05 nb_pixel_total : 383 time to create 1 rle with old method : 0.0005075931549072266 length of segment : 32 time for calcul the mask position with numpy : 3.7670135498046875e-05 nb_pixel_total : 496 time to create 1 rle with old method : 0.0006716251373291016 length of segment : 28 time for calcul the mask position with numpy : 4.458427429199219e-05 nb_pixel_total : 673 time to create 1 rle with old method : 0.0009450912475585938 length of segment : 29 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 146.44766 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 11 time for calcul the mask position with numpy : 6.413459777832031e-05 nb_pixel_total : 257 time to create 1 rle with old method : 0.0005078315734863281 length of segment : 50 time for calcul the mask position with numpy : 6.270408630371094e-05 nb_pixel_total : 491 time to create 1 rle with old method : 0.0009298324584960938 length of segment : 28 time for calcul the mask position with numpy : 5.269050598144531e-05 nb_pixel_total : 371 time to create 1 rle with old method : 0.0007588863372802734 length of segment : 20 time for calcul the mask position with numpy : 4.8160552978515625e-05 nb_pixel_total : 107 time to create 1 rle with old method : 0.0002486705780029297 length of segment : 8 time for calcul the mask position with numpy : 6.437301635742188e-05 nb_pixel_total : 1115 time to create 1 rle with old method : 0.0019817352294921875 length of segment : 48 time for calcul the mask position with numpy : 0.0006527900695800781 nb_pixel_total : 40928 time to create 1 rle with old method : 0.04931521415710449 length of segment : 266 time for calcul the mask position with numpy : 6.4849853515625e-05 nb_pixel_total : 902 time to create 1 rle with old method : 0.0012326240539550781 length of segment : 41 time for calcul the mask position with numpy : 4.363059997558594e-05 nb_pixel_total : 399 time to create 1 rle with old method : 0.0005917549133300781 length of segment : 25 time for calcul the mask position with numpy : 0.0005877017974853516 nb_pixel_total : 45969 time to create 1 rle with old method : 0.08381295204162598 length of segment : 239 time for calcul the mask position with numpy : 4.935264587402344e-05 nb_pixel_total : 397 time to create 1 rle with old method : 0.0006022453308105469 length of segment : 25 time for calcul the mask position with numpy : 3.409385681152344e-05 nb_pixel_total : 360 time to create 1 rle with old method : 0.0005075931549072266 length of segment : 24 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 141.97500 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 4.315376281738281e-05 nb_pixel_total : 157 time to create 1 rle with old method : 0.0003032684326171875 length of segment : 12 time for calcul the mask position with numpy : 0.0004513263702392578 nb_pixel_total : 28603 time to create 1 rle with old method : 0.033838510513305664 length of segment : 246 time for calcul the mask position with numpy : 6.628036499023438e-05 nb_pixel_total : 135 time to create 1 rle with old method : 0.0002791881561279297 length of segment : 22 Detection mask done ! Trying to reset tf kernel 3822408 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 4465 tf kernel not reseted sub process len(results) : 208 len(list_Values) 208 None max_time_sub_proc : 3600 parent process len(results) : 0 len(list_Values) 208 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 9754 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Catched exception ! Connect or reconnect ! Number saved : None batch 1 Loaded 208 chid ids of type : 4228 Number RLEs to save : 11954 save missing photos in datou_result : time spend for datou_step_exec : 22.98378562927246 time spend to save output : 0.9379496574401855 total time spend for step 1 : 23.921735286712646 step2:brightness Sun Feb 9 16:20:53 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1739114428_3821838_1335899451_80b2f7ff5d2b09fac3777b7ec2d5a027.jpg treat image : temp/1739114428_3821838_1335899449_f3d1446e4522ffc6b89fe1f7e25d4b2f.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 2 time used for this insertion : 0.01085209846496582 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 2 time used for this insertion : 0.011191606521606445 save missing photos in datou_result : time spend for datou_step_exec : 0.4883413314819336 time spend to save output : 0.026443004608154297 total time spend for step 2 : 0.5147843360900879 step3:blur_detection Sun Feb 9 16:20:53 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1739114428_3821838_1335899451_80b2f7ff5d2b09fac3777b7ec2d5a027.jpg resize: (1080, 1920) 1335899451 -7.3465638826423385 treat image : temp/1739114428_3821838_1335899449_f3d1446e4522ffc6b89fe1f7e25d4b2f.jpg resize: (1080, 1920) 1335899449 -7.343381079546548 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 2 time used for this insertion : 0.01072382926940918 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 2 time used for this insertion : 0.011029720306396484 save missing photos in datou_result : time spend for datou_step_exec : 1.6132092475891113 time spend to save output : 0.02599191665649414 total time spend for step 3 : 1.6392011642456055 step4:rle_unique_nms_with_priority Sun Feb 9 16:20:55 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 208 chid ids of type : 4228 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 99 nb_hashtags : 8 time to prepare the origin masks : 1.1761534214019775 time for calcul the mask position with numpy : 0.02457118034362793 nb_pixel_total : 1711800 time to create 1 rle with new method : 0.08692765235900879 time for calcul the mask position with numpy : 0.008392333984375 nb_pixel_total : 146 time to create 1 rle with old method : 0.0002238750457763672 time for calcul the mask position with numpy : 0.00834512710571289 nb_pixel_total : 2351 time to create 1 rle with old method : 0.002813100814819336 time for calcul the mask position with numpy : 0.008395671844482422 nb_pixel_total : 23977 time to create 1 rle with old method : 0.026532888412475586 time for calcul the mask position with numpy : 0.008387565612792969 nb_pixel_total : 836 time to create 1 rle with old method : 0.0009343624114990234 time for calcul the mask position with numpy : 0.008390665054321289 nb_pixel_total : 43 time to create 1 rle with old method : 8.296966552734375e-05 time for calcul the mask position with numpy : 0.008386850357055664 nb_pixel_total : 129 time to create 1 rle with old method : 0.00016999244689941406 time for calcul the mask position with numpy : 0.008500099182128906 nb_pixel_total : 13 time to create 1 rle with old method : 3.62396240234375e-05 time for calcul the mask position with numpy : 0.008779764175415039 nb_pixel_total : 24 time to create 1 rle with old method : 7.462501525878906e-05 time for calcul the mask position with numpy : 0.008891105651855469 nb_pixel_total : 370 time to create 1 rle with old method : 0.0005671977996826172 time for calcul the mask position with numpy : 0.010645151138305664 nb_pixel_total : 237 time to create 1 rle with old method : 0.0003714561462402344 time for calcul the mask position with numpy : 0.010421514511108398 nb_pixel_total : 2422 time to create 1 rle with old method : 0.0029032230377197266 time for calcul the mask position with numpy : 0.008352518081665039 nb_pixel_total : 356 time to create 1 rle with old method : 0.0004410743713378906 time for calcul the mask position with numpy : 0.00833749771118164 nb_pixel_total : 3039 time to create 1 rle with old method : 0.005313873291015625 time for calcul the mask position with numpy : 0.008174896240234375 nb_pixel_total : 438 time to create 1 rle with old method : 0.0005271434783935547 time for calcul the mask position with numpy : 0.008256673812866211 nb_pixel_total : 1773 time to create 1 rle with old method : 0.0021114349365234375 time for calcul the mask position with numpy : 0.008265972137451172 nb_pixel_total : 6821 time to create 1 rle with old method : 0.007578372955322266 time for calcul the mask position with numpy : 0.008380413055419922 nb_pixel_total : 9563 time to create 1 rle with old method : 0.011019229888916016 time for calcul the mask position with numpy : 0.008436441421508789 nb_pixel_total : 15 time to create 1 rle with old method : 4.076957702636719e-05 time for calcul the mask position with numpy : 0.00841975212097168 nb_pixel_total : 129 time to create 1 rle with old method : 0.00016307830810546875 time for calcul the mask position with numpy : 0.008346796035766602 nb_pixel_total : 643 time to create 1 rle with old method : 0.0007266998291015625 time for calcul the mask position with numpy : 0.008354425430297852 nb_pixel_total : 173 time to create 1 rle with old method : 0.00020503997802734375 time for calcul the mask position with numpy : 0.008341550827026367 nb_pixel_total : 4460 time to create 1 rle with old method : 0.0051975250244140625 time for calcul the mask position with numpy : 0.008458137512207031 nb_pixel_total : 68436 time to create 1 rle with old method : 0.0747842788696289 time for calcul the mask position with numpy : 0.008470773696899414 nb_pixel_total : 1061 time to create 1 rle with old method : 0.0012030601501464844 time for calcul the mask position with numpy : 0.00837564468383789 nb_pixel_total : 417 time to create 1 rle with old method : 0.0004868507385253906 time for calcul the mask position with numpy : 0.008373737335205078 nb_pixel_total : 88 time to create 1 rle with old method : 0.00012135505676269531 time for calcul the mask position with numpy : 0.008358955383300781 nb_pixel_total : 168 time to create 1 rle with old method : 0.00021314620971679688 time for calcul the mask position with numpy : 0.008305788040161133 nb_pixel_total : 3594 time to create 1 rle with old method : 0.00408172607421875 time for calcul the mask position with numpy : 0.008329391479492188 nb_pixel_total : 695 time to create 1 rle with old method : 0.0007975101470947266 time for calcul the mask position with numpy : 0.008184671401977539 nb_pixel_total : 110 time to create 1 rle with old method : 0.00012874603271484375 time for calcul the mask position with numpy : 0.008072376251220703 nb_pixel_total : 554 time to create 1 rle with old method : 0.0006666183471679688 time for calcul the mask position with numpy : 0.008296489715576172 nb_pixel_total : 4967 time to create 1 rle with old method : 0.006083488464355469 time for calcul the mask position with numpy : 0.008556604385375977 nb_pixel_total : 422 time to create 1 rle with old method : 0.000530242919921875 time for calcul the mask position with numpy : 0.008435726165771484 nb_pixel_total : 660 time to create 1 rle with old method : 0.0007722377777099609 time for calcul the mask position with numpy : 0.008404970169067383 nb_pixel_total : 53 time to create 1 rle with old method : 9.822845458984375e-05 time for calcul the mask position with numpy : 0.008356332778930664 nb_pixel_total : 1692 time to create 1 rle with old method : 0.001993417739868164 time for calcul the mask position with numpy : 0.008374214172363281 nb_pixel_total : 851 time to create 1 rle with old method : 0.0009930133819580078 time for calcul the mask position with numpy : 0.008463382720947266 nb_pixel_total : 654 time to create 1 rle with old method : 0.0007925033569335938 time for calcul the mask position with numpy : 0.008385419845581055 nb_pixel_total : 885 time to create 1 rle with old method : 0.0010347366333007812 time for calcul the mask position with numpy : 0.008364200592041016 nb_pixel_total : 507 time to create 1 rle with old method : 0.0006198883056640625 time for calcul the mask position with numpy : 0.008366823196411133 nb_pixel_total : 114 time to create 1 rle with old method : 0.0001575946807861328 time for calcul the mask position with numpy : 0.008390188217163086 nb_pixel_total : 47 time to create 1 rle with old method : 6.699562072753906e-05 time for calcul the mask position with numpy : 0.008536338806152344 nb_pixel_total : 121 time to create 1 rle with old method : 0.00016236305236816406 time for calcul the mask position with numpy : 0.00851750373840332 nb_pixel_total : 405 time to create 1 rle with old method : 0.0005838871002197266 time for calcul the mask position with numpy : 0.008464336395263672 nb_pixel_total : 1313 time to create 1 rle with old method : 0.0015735626220703125 time for calcul the mask position with numpy : 0.008565902709960938 nb_pixel_total : 290 time to create 1 rle with old method : 0.0003650188446044922 time for calcul the mask position with numpy : 0.008572101593017578 nb_pixel_total : 140 time to create 1 rle with old method : 0.00017452239990234375 time for calcul the mask position with numpy : 0.008388996124267578 nb_pixel_total : 394 time to create 1 rle with old method : 0.00047278404235839844 time for calcul the mask position with numpy : 0.008379459381103516 nb_pixel_total : 2321 time to create 1 rle with old method : 0.0026628971099853516 time for calcul the mask position with numpy : 0.008380651473999023 nb_pixel_total : 846 time to create 1 rle with old method : 0.0009813308715820312 time for calcul the mask position with numpy : 0.008502721786499023 nb_pixel_total : 37 time to create 1 rle with old method : 7.605552673339844e-05 time for calcul the mask position with numpy : 0.008528709411621094 nb_pixel_total : 1803 time to create 1 rle with old method : 0.0020341873168945312 time for calcul the mask position with numpy : 0.00836181640625 nb_pixel_total : 209 time to create 1 rle with old method : 0.00029206275939941406 time for calcul the mask position with numpy : 0.008365869522094727 nb_pixel_total : 457 time to create 1 rle with old method : 0.0005369186401367188 time for calcul the mask position with numpy : 0.00835108757019043 nb_pixel_total : 162 time to create 1 rle with old method : 0.0001952648162841797 time for calcul the mask position with numpy : 0.008382081985473633 nb_pixel_total : 291 time to create 1 rle with old method : 0.00035762786865234375 time for calcul the mask position with numpy : 0.008483648300170898 nb_pixel_total : 36 time to create 1 rle with old method : 5.6743621826171875e-05 time for calcul the mask position with numpy : 0.008784770965576172 nb_pixel_total : 106420 time to create 1 rle with old method : 0.11759662628173828 time for calcul the mask position with numpy : 0.00810384750366211 nb_pixel_total : 276 time to create 1 rle with old method : 0.0003256797790527344 time for calcul the mask position with numpy : 0.008106231689453125 nb_pixel_total : 357 time to create 1 rle with old method : 0.00043702125549316406 time for calcul the mask position with numpy : 0.008200407028198242 nb_pixel_total : 43049 time to create 1 rle with old method : 0.06281828880310059 time for calcul the mask position with numpy : 0.010862350463867188 nb_pixel_total : 470 time to create 1 rle with old method : 0.0011873245239257812 time for calcul the mask position with numpy : 0.01124119758605957 nb_pixel_total : 692 time to create 1 rle with old method : 0.0011756420135498047 time for calcul the mask position with numpy : 0.009125709533691406 nb_pixel_total : 2141 time to create 1 rle with old method : 0.002644062042236328 time for calcul the mask position with numpy : 0.008424043655395508 nb_pixel_total : 937 time to create 1 rle with old method : 0.0010843276977539062 time for calcul the mask position with numpy : 0.008420705795288086 nb_pixel_total : 9 time to create 1 rle with old method : 3.719329833984375e-05 time for calcul the mask position with numpy : 0.0099334716796875 nb_pixel_total : 102 time to create 1 rle with old method : 0.00018215179443359375 time for calcul the mask position with numpy : 0.008505105972290039 nb_pixel_total : 218 time to create 1 rle with old method : 0.0002741813659667969 time for calcul the mask position with numpy : 0.008513927459716797 nb_pixel_total : 160 time to create 1 rle with old method : 0.0003859996795654297 time for calcul the mask position with numpy : 0.008965015411376953 nb_pixel_total : 41669 time to create 1 rle with old method : 0.047121286392211914 time for calcul the mask position with numpy : 0.008417844772338867 nb_pixel_total : 277 time to create 1 rle with old method : 0.0003368854522705078 time for calcul the mask position with numpy : 0.008421182632446289 nb_pixel_total : 469 time to create 1 rle with old method : 0.0005855560302734375 time for calcul the mask position with numpy : 0.008584737777709961 nb_pixel_total : 580 time to create 1 rle with old method : 0.0008368492126464844 time for calcul the mask position with numpy : 0.008198022842407227 nb_pixel_total : 981 time to create 1 rle with old method : 0.0011467933654785156 time for calcul the mask position with numpy : 0.008399248123168945 nb_pixel_total : 174 time to create 1 rle with old method : 0.00021648406982421875 time for calcul the mask position with numpy : 0.008566141128540039 nb_pixel_total : 378 time to create 1 rle with old method : 0.00046753883361816406 time for calcul the mask position with numpy : 0.008467912673950195 nb_pixel_total : 1043 time to create 1 rle with old method : 0.001207113265991211 time for calcul the mask position with numpy : 0.008404016494750977 nb_pixel_total : 164 time to create 1 rle with old method : 0.0001990795135498047 time for calcul the mask position with numpy : 0.00838470458984375 nb_pixel_total : 72 time to create 1 rle with old method : 9.560585021972656e-05 time for calcul the mask position with numpy : 0.008382081985473633 nb_pixel_total : 499 time to create 1 rle with old method : 0.0005776882171630859 time for calcul the mask position with numpy : 0.009260892868041992 nb_pixel_total : 388 time to create 1 rle with old method : 0.0004582405090332031 time for calcul the mask position with numpy : 0.008398771286010742 nb_pixel_total : 584 time to create 1 rle with old method : 0.0006837844848632812 time for calcul the mask position with numpy : 0.008423566818237305 nb_pixel_total : 740 time to create 1 rle with old method : 0.0008692741394042969 time for calcul the mask position with numpy : 0.008403301239013672 nb_pixel_total : 473 time to create 1 rle with old method : 0.0005855560302734375 time for calcul the mask position with numpy : 0.008391618728637695 nb_pixel_total : 220 time to create 1 rle with old method : 0.0002598762512207031 time for calcul the mask position with numpy : 0.008417606353759766 nb_pixel_total : 306 time to create 1 rle with old method : 0.00036597251892089844 time for calcul the mask position with numpy : 0.00841832160949707 nb_pixel_total : 1244 time to create 1 rle with old method : 0.0014986991882324219 time for calcul the mask position with numpy : 0.008944034576416016 nb_pixel_total : 366 time to create 1 rle with old method : 0.0004439353942871094 time for calcul the mask position with numpy : 0.008599042892456055 nb_pixel_total : 820 time to create 1 rle with old method : 0.0010900497436523438 time for calcul the mask position with numpy : 0.008756399154663086 nb_pixel_total : 742 time to create 1 rle with old method : 0.0010073184967041016 time for calcul the mask position with numpy : 0.00860142707824707 nb_pixel_total : 370 time to create 1 rle with old method : 0.0004742145538330078 time for calcul the mask position with numpy : 0.008434534072875977 nb_pixel_total : 78 time to create 1 rle with old method : 0.0001246929168701172 time for calcul the mask position with numpy : 0.008440971374511719 nb_pixel_total : 222 time to create 1 rle with old method : 0.000270843505859375 time for calcul the mask position with numpy : 0.008428335189819336 nb_pixel_total : 233 time to create 1 rle with old method : 0.0002906322479248047 time for calcul the mask position with numpy : 0.008557558059692383 nb_pixel_total : 305 time to create 1 rle with old method : 0.0005819797515869141 time for calcul the mask position with numpy : 0.008438348770141602 nb_pixel_total : 379 time to create 1 rle with old method : 0.0004489421844482422 time for calcul the mask position with numpy : 0.008397579193115234 nb_pixel_total : 109 time to create 1 rle with old method : 0.00015306472778320312 time for calcul the mask position with numpy : 0.008280277252197266 nb_pixel_total : 214 time to create 1 rle with old method : 0.0002636909484863281 time for calcul the mask position with numpy : 0.008358478546142578 nb_pixel_total : 112 time to create 1 rle with old method : 0.00013446807861328125 create new chi : 1.3918118476867676 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.006469011306762695 batch 1 Loaded 100 chid ids of type : 4230 Number RLEs to save : 11714 TO DO : save crop sub photo not yet done ! save time : 0.6812489032745361 nb_obj : 106 nb_hashtags : 7 time to prepare the origin masks : 2.0328469276428223 time for calcul the mask position with numpy : 0.02843952178955078 nb_pixel_total : 1729230 time to create 1 rle with new method : 1.1539490222930908 time for calcul the mask position with numpy : 0.008082389831542969 nb_pixel_total : 1608 time to create 1 rle with old method : 0.002381563186645508 time for calcul the mask position with numpy : 0.007776498794555664 nb_pixel_total : 81 time to create 1 rle with old method : 0.00012135505676269531 time for calcul the mask position with numpy : 0.008651971817016602 nb_pixel_total : 499 time to create 1 rle with old method : 0.0009031295776367188 time for calcul the mask position with numpy : 0.009371757507324219 nb_pixel_total : 32 time to create 1 rle with old method : 8.487701416015625e-05 time for calcul the mask position with numpy : 0.007899284362792969 nb_pixel_total : 666 time to create 1 rle with old method : 0.0009443759918212891 time for calcul the mask position with numpy : 0.007605791091918945 nb_pixel_total : 310 time to create 1 rle with old method : 0.0004925727844238281 time for calcul the mask position with numpy : 0.008117437362670898 nb_pixel_total : 131 time to create 1 rle with old method : 0.0002200603485107422 time for calcul the mask position with numpy : 0.007224559783935547 nb_pixel_total : 196 time to create 1 rle with old method : 0.00035309791564941406 time for calcul the mask position with numpy : 0.007025480270385742 nb_pixel_total : 412 time to create 1 rle with old method : 0.00057220458984375 time for calcul the mask position with numpy : 0.008191347122192383 nb_pixel_total : 506 time to create 1 rle with old method : 0.0006883144378662109 time for calcul the mask position with numpy : 0.007179737091064453 nb_pixel_total : 7 time to create 1 rle with old method : 4.4345855712890625e-05 time for calcul the mask position with numpy : 0.007449626922607422 nb_pixel_total : 34 time to create 1 rle with old method : 0.00010561943054199219 time for calcul the mask position with numpy : 0.00859689712524414 nb_pixel_total : 879 time to create 1 rle with old method : 0.0010852813720703125 time for calcul the mask position with numpy : 0.007870912551879883 nb_pixel_total : 279 time to create 1 rle with old method : 0.0004162788391113281 time for calcul the mask position with numpy : 0.007152080535888672 nb_pixel_total : 21 time to create 1 rle with old method : 6.198883056640625e-05 time for calcul the mask position with numpy : 0.008269548416137695 nb_pixel_total : 45121 time to create 1 rle with old method : 0.0562589168548584 time for calcul the mask position with numpy : 0.006946563720703125 nb_pixel_total : 2423 time to create 1 rle with old method : 0.0028362274169921875 time for calcul the mask position with numpy : 0.006628990173339844 nb_pixel_total : 23484 time to create 1 rle with old method : 0.02676844596862793 time for calcul the mask position with numpy : 0.006561756134033203 nb_pixel_total : 497 time to create 1 rle with old method : 0.0006794929504394531 time for calcul the mask position with numpy : 0.006462812423706055 nb_pixel_total : 8598 time to create 1 rle with old method : 0.009630203247070312 time for calcul the mask position with numpy : 0.00655674934387207 nb_pixel_total : 4893 time to create 1 rle with old method : 0.0057525634765625 time for calcul the mask position with numpy : 0.006512165069580078 nb_pixel_total : 220 time to create 1 rle with old method : 0.00030922889709472656 time for calcul the mask position with numpy : 0.009922504425048828 nb_pixel_total : 827 time to create 1 rle with old method : 0.001004934310913086 time for calcul the mask position with numpy : 0.006779909133911133 nb_pixel_total : 453 time to create 1 rle with old method : 0.0005686283111572266 time for calcul the mask position with numpy : 0.006469011306762695 nb_pixel_total : 5931 time to create 1 rle with old method : 0.006688833236694336 time for calcul the mask position with numpy : 0.00647425651550293 nb_pixel_total : 1741 time to create 1 rle with old method : 0.002053976058959961 time for calcul the mask position with numpy : 0.006551980972290039 nb_pixel_total : 7367 time to create 1 rle with old method : 0.008475542068481445 time for calcul the mask position with numpy : 0.00786590576171875 nb_pixel_total : 131 time to create 1 rle with old method : 0.00017595291137695312 time for calcul the mask position with numpy : 0.00691533088684082 nb_pixel_total : 670 time to create 1 rle with old method : 0.0008218288421630859 time for calcul the mask position with numpy : 0.008429288864135742 nb_pixel_total : 837 time to create 1 rle with old method : 0.0010347366333007812 time for calcul the mask position with numpy : 0.007573366165161133 nb_pixel_total : 214 time to create 1 rle with old method : 0.00037932395935058594 time for calcul the mask position with numpy : 0.007677316665649414 nb_pixel_total : 69 time to create 1 rle with old method : 0.00015211105346679688 time for calcul the mask position with numpy : 0.007021188735961914 nb_pixel_total : 12741 time to create 1 rle with old method : 0.013791561126708984 time for calcul the mask position with numpy : 0.006397724151611328 nb_pixel_total : 221 time to create 1 rle with old method : 0.0002837181091308594 time for calcul the mask position with numpy : 0.006319522857666016 nb_pixel_total : 774 time to create 1 rle with old method : 0.0012857913970947266 time for calcul the mask position with numpy : 0.007677793502807617 nb_pixel_total : 41 time to create 1 rle with old method : 9.1552734375e-05 time for calcul the mask position with numpy : 0.007383108139038086 nb_pixel_total : 828 time to create 1 rle with old method : 0.0013353824615478516 time for calcul the mask position with numpy : 0.007321834564208984 nb_pixel_total : 4573 time to create 1 rle with old method : 0.00536799430847168 time for calcul the mask position with numpy : 0.006845235824584961 nb_pixel_total : 29 time to create 1 rle with old method : 8.320808410644531e-05 time for calcul the mask position with numpy : 0.007149696350097656 nb_pixel_total : 5220 time to create 1 rle with old method : 0.005780458450317383 time for calcul the mask position with numpy : 0.0067980289459228516 nb_pixel_total : 639 time to create 1 rle with old method : 0.0007965564727783203 time for calcul the mask position with numpy : 0.006630659103393555 nb_pixel_total : 454 time to create 1 rle with old method : 0.0005497932434082031 time for calcul the mask position with numpy : 0.006860256195068359 nb_pixel_total : 1956 time to create 1 rle with old method : 0.0022847652435302734 time for calcul the mask position with numpy : 0.006550788879394531 nb_pixel_total : 891 time to create 1 rle with old method : 0.001102447509765625 time for calcul the mask position with numpy : 0.0070056915283203125 nb_pixel_total : 2030 time to create 1 rle with old method : 0.0023555755615234375 time for calcul the mask position with numpy : 0.007421970367431641 nb_pixel_total : 864 time to create 1 rle with old method : 0.0009717941284179688 time for calcul the mask position with numpy : 0.0072209835052490234 nb_pixel_total : 1163 time to create 1 rle with old method : 0.0013833045959472656 time for calcul the mask position with numpy : 0.006482601165771484 nb_pixel_total : 284 time to create 1 rle with old method : 0.00035881996154785156 time for calcul the mask position with numpy : 0.006910800933837891 nb_pixel_total : 129 time to create 1 rle with old method : 0.0001735687255859375 time for calcul the mask position with numpy : 0.007063388824462891 nb_pixel_total : 43 time to create 1 rle with old method : 0.00010561943054199219 time for calcul the mask position with numpy : 0.008883953094482422 nb_pixel_total : 102 time to create 1 rle with old method : 0.00014019012451171875 time for calcul the mask position with numpy : 0.009021282196044922 nb_pixel_total : 417 time to create 1 rle with old method : 0.0004985332489013672 time for calcul the mask position with numpy : 0.007223606109619141 nb_pixel_total : 1607 time to create 1 rle with old method : 0.0018525123596191406 time for calcul the mask position with numpy : 0.007716655731201172 nb_pixel_total : 77 time to create 1 rle with old method : 0.0001437664031982422 time for calcul the mask position with numpy : 0.009979724884033203 nb_pixel_total : 239 time to create 1 rle with old method : 0.00029778480529785156 time for calcul the mask position with numpy : 0.009496688842773438 nb_pixel_total : 314 time to create 1 rle with old method : 0.000400543212890625 time for calcul the mask position with numpy : 0.009306907653808594 nb_pixel_total : 140 time to create 1 rle with old method : 0.0001964569091796875 time for calcul the mask position with numpy : 0.008533477783203125 nb_pixel_total : 359 time to create 1 rle with old method : 0.0004363059997558594 time for calcul the mask position with numpy : 0.0067844390869140625 nb_pixel_total : 78 time to create 1 rle with old method : 0.00014281272888183594 time for calcul the mask position with numpy : 0.008583545684814453 nb_pixel_total : 797 time to create 1 rle with old method : 0.0009343624114990234 time for calcul the mask position with numpy : 0.010017156600952148 nb_pixel_total : 24 time to create 1 rle with old method : 7.081031799316406e-05 time for calcul the mask position with numpy : 0.00985574722290039 nb_pixel_total : 1476 time to create 1 rle with old method : 0.0017344951629638672 time for calcul the mask position with numpy : 0.009665966033935547 nb_pixel_total : 451 time to create 1 rle with old method : 0.0004966259002685547 time for calcul the mask position with numpy : 0.010308265686035156 nb_pixel_total : 106136 time to create 1 rle with old method : 0.1144566535949707 time for calcul the mask position with numpy : 0.006785392761230469 nb_pixel_total : 353 time to create 1 rle with old method : 0.0004355907440185547 time for calcul the mask position with numpy : 0.006125211715698242 nb_pixel_total : 1048 time to create 1 rle with old method : 0.0011830329895019531 time for calcul the mask position with numpy : 0.00614619255065918 nb_pixel_total : 70 time to create 1 rle with old method : 0.00011324882507324219 time for calcul the mask position with numpy : 0.0073506832122802734 nb_pixel_total : 40928 time to create 1 rle with old method : 0.04399895668029785 time for calcul the mask position with numpy : 0.006163835525512695 nb_pixel_total : 5970 time to create 1 rle with old method : 0.0069315433502197266 time for calcul the mask position with numpy : 0.0059452056884765625 nb_pixel_total : 101 time to create 1 rle with old method : 0.00012803077697753906 time for calcul the mask position with numpy : 0.00597691535949707 nb_pixel_total : 135 time to create 1 rle with old method : 0.00019359588623046875 time for calcul the mask position with numpy : 0.005885124206542969 nb_pixel_total : 272 time to create 1 rle with old method : 0.0003261566162109375 time for calcul the mask position with numpy : 0.006333112716674805 nb_pixel_total : 247 time to create 1 rle with old method : 0.0002894401550292969 time for calcul the mask position with numpy : 0.006487846374511719 nb_pixel_total : 28603 time to create 1 rle with old method : 0.03087759017944336 time for calcul the mask position with numpy : 0.006487131118774414 nb_pixel_total : 544 time to create 1 rle with old method : 0.0006520748138427734 time for calcul the mask position with numpy : 0.006352663040161133 nb_pixel_total : 251 time to create 1 rle with old method : 0.00029778480529785156 time for calcul the mask position with numpy : 0.006552934646606445 nb_pixel_total : 383 time to create 1 rle with old method : 0.0004584789276123047 time for calcul the mask position with numpy : 0.006403207778930664 nb_pixel_total : 157 time to create 1 rle with old method : 0.00019884109497070312 time for calcul the mask position with numpy : 0.00642085075378418 nb_pixel_total : 276 time to create 1 rle with old method : 0.0003533363342285156 time for calcul the mask position with numpy : 0.006348371505737305 nb_pixel_total : 194 time to create 1 rle with old method : 0.00023317337036132812 time for calcul the mask position with numpy : 0.006339311599731445 nb_pixel_total : 243 time to create 1 rle with old method : 0.00030612945556640625 time for calcul the mask position with numpy : 0.006974935531616211 nb_pixel_total : 1115 time to create 1 rle with old method : 0.001294851303100586 time for calcul the mask position with numpy : 0.006608247756958008 nb_pixel_total : 257 time to create 1 rle with old method : 0.00031948089599609375 time for calcul the mask position with numpy : 0.006480693817138672 nb_pixel_total : 204 time to create 1 rle with old method : 0.0002617835998535156 time for calcul the mask position with numpy : 0.006406545639038086 nb_pixel_total : 446 time to create 1 rle with old method : 0.0005567073822021484 time for calcul the mask position with numpy : 0.006455898284912109 nb_pixel_total : 399 time to create 1 rle with old method : 0.00048351287841796875 time for calcul the mask position with numpy : 0.0065042972564697266 nb_pixel_total : 18 time to create 1 rle with old method : 5.745887756347656e-05 time for calcul the mask position with numpy : 0.006430149078369141 nb_pixel_total : 639 time to create 1 rle with old method : 0.0007555484771728516 time for calcul the mask position with numpy : 0.006382465362548828 nb_pixel_total : 609 time to create 1 rle with old method : 0.0007977485656738281 time for calcul the mask position with numpy : 0.0062983036041259766 nb_pixel_total : 131 time to create 1 rle with old method : 0.0001773834228515625 time for calcul the mask position with numpy : 0.006343364715576172 nb_pixel_total : 491 time to create 1 rle with old method : 0.0005843639373779297 time for calcul the mask position with numpy : 0.006390571594238281 nb_pixel_total : 902 time to create 1 rle with old method : 0.0010445117950439453 time for calcul the mask position with numpy : 0.006444215774536133 nb_pixel_total : 354 time to create 1 rle with old method : 0.000415802001953125 time for calcul the mask position with numpy : 0.0067827701568603516 nb_pixel_total : 292 time to create 1 rle with old method : 0.0003628730773925781 time for calcul the mask position with numpy : 0.0069217681884765625 nb_pixel_total : 376 time to create 1 rle with old method : 0.0004620552062988281 time for calcul the mask position with numpy : 0.007066965103149414 nb_pixel_total : 124 time to create 1 rle with old method : 0.00016760826110839844 time for calcul the mask position with numpy : 0.00703740119934082 nb_pixel_total : 1567 time to create 1 rle with old method : 0.001837015151977539 time for calcul the mask position with numpy : 0.00653386116027832 nb_pixel_total : 228 time to create 1 rle with old method : 0.0002970695495605469 time for calcul the mask position with numpy : 0.006468534469604492 nb_pixel_total : 710 time to create 1 rle with old method : 0.00083160400390625 time for calcul the mask position with numpy : 0.006649017333984375 nb_pixel_total : 496 time to create 1 rle with old method : 0.0006072521209716797 time for calcul the mask position with numpy : 0.00839376449584961 nb_pixel_total : 5 time to create 1 rle with old method : 1.3828277587890625e-05 time for calcul the mask position with numpy : 0.008365631103515625 nb_pixel_total : 371 time to create 1 rle with old method : 0.00043582916259765625 time for calcul the mask position with numpy : 0.009424448013305664 nb_pixel_total : 288 time to create 1 rle with old method : 0.0004317760467529297 time for calcul the mask position with numpy : 0.009552955627441406 nb_pixel_total : 59 time to create 1 rle with old method : 0.00012612342834472656 time for calcul the mask position with numpy : 0.00837087631225586 nb_pixel_total : 173 time to create 1 rle with old method : 0.00020503997802734375 time for calcul the mask position with numpy : 0.008202314376831055 nb_pixel_total : 107 time to create 1 rle with old method : 0.00012302398681640625 create new chi : 2.361255645751953 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.0013453960418701172 batch 1 Loaded 107 chid ids of type : 4230 Number RLEs to save : 11527 TO DO : save crop sub photo not yet done ! save time : 0.6608951091766357 map_output_result : {1335899451: (0.0, 'Should be the crop_list due to order', 0.0), 1335899449: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1335899451, 1335899449] Looping around the photos to save general results len do output : 2 /1335899451.Didn't retrieve data . /1335899449.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2569149') ('4234', '20363428', '1335899451', None, None, None, None, None, '2569149') ('4234', None, None, None, None, None, None, None, '2569149') ('4234', '20363428', '1335899449', None, None, None, None, None, '2569149') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.013643741607666016 save_final save missing photos in datou_result : time spend for datou_step_exec : 8.691602230072021 time spend to save output : 0.022681236267089844 total time spend for step 4 : 8.714283466339111 step5:crop_condition Sun Feb 9 16:21:03 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4230 Loading chi in step crop for list_pids : 2 ! batch 1 Loaded 207 chid ids of type : 4230 begin to crop the class : papier param for this class : {'min_score': 0.6} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 70 About to insert : list_path_to_insert length 70 new photo from crops ! About to upload 70 photos upload in portfolio : 4869462 init cache_photo without model_param we have 70 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739114465_3821838 we have uploaded 70 photos in the portfolio 4869462 time of upload the photos Elapsed time : 13.901854991912842 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.6} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 33 About to insert : list_path_to_insert length 33 new photo from crops ! About to upload 33 photos upload in portfolio : 4869462 init cache_photo without model_param we have 33 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739114479_3821838 we have uploaded 33 photos in the portfolio 4869462 time of upload the photos Elapsed time : 9.966062545776367 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.6} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.6} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! About to upload 8 photos upload in portfolio : 4869462 init cache_photo without model_param we have 8 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739114490_3821838 we have uploaded 8 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.8095390796661377 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.6} filtre for class : autre hashtag_id of this class : 494826614 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 25 About to insert : list_path_to_insert length 25 new photo from crops ! About to upload 25 photos upload in portfolio : 4869462 init cache_photo without model_param we have 25 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739114494_3821838 we have uploaded 25 photos in the portfolio 4869462 time of upload the photos Elapsed time : 5.910908937454224 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.6} filtre for class : pehd hashtag_id of this class : 628944319 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739114500_3821838 we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.50999116897583 we have finished the crop for the class : pehd begin to crop the class : pet_fonce param for this class : {'min_score': 0.6} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1335899451, 1335899449] Looping around the photos to save general results len do output : 138 /1336209811Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209812Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209813Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209814Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209815Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209817Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209818Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209819Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209820Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209821Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209822Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209823Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209824Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209825Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209826Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209827Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209828Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209829Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209830Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209831Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209832Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209833Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209834Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209835Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209836Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209837Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209838Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209839Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209840Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209841Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209842Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209843Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209844Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209845Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209846Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209847Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209848Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209849Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209850Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209851Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209852Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209853Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209854Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209855Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209856Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209857Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209858Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209859Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209860Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209861Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209863Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209865Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209867Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209869Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209871Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209873Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209875Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209877Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209879Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209881Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209883Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209885Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209887Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209889Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209891Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209893Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209895Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209897Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209899Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209901Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209955Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209956Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209957Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209958Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209959Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209960Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209961Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209962Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209963Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209964Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209966Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209967Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209968Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209969Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209970Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209971Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209972Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209973Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209974Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209976Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209977Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209978Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209979Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209981Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209982Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209983Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209984Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209985Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209986Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209987Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209988Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209989Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209990Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209992Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209993Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209994Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209995Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209996Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209997Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209998Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336209999Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210003Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210004Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210005Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210006Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210007Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210008Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210009Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210010Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210011Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210012Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210013Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210014Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210015Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210016Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210017Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210018Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210019Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210020Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210021Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210022Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210023Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210024Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210025Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210026Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210027Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210028Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336210029Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2569149') ('4234', '20363428', '1335899451', None, None, None, None, None, '2569149') ('4234', None, None, None, None, None, None, None, '2569149') ('4234', '20363428', '1335899449', None, None, None, None, None, '2569149') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 416 time used for this insertion : 0.0342099666595459 save_final save missing photos in datou_result : time spend for datou_step_exec : 37.9056282043457 time spend to save output : 0.03920865058898926 total time spend for step 5 : 37.94483685493469 step6:thcl Sun Feb 9 16:21:41 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! we are using the classfication for only one thcl 3237 time to import caffe and check if the image exist : 0.005338430404663086 time to convert the images to numpy array : 0.033841609954833984 time to import caffe and check if the image exist : 0.010960102081298828 time to convert the images to numpy array : 0.034948110580444336 time to import caffe and check if the image exist : 0.005559682846069336 time to convert the images to numpy array : 0.04008173942565918 time to import caffe and check if the image exist : 0.006990671157836914 time to convert the images to numpy array : 0.04073500633239746 time to import caffe and check if the image exist : 0.012667417526245117 time to convert the images to numpy array : 0.03703117370605469 time to import caffe and check if the image exist : 0.008303642272949219 time to convert the images to numpy array : 0.04304647445678711 time to import caffe and check if the image exist : 0.012854576110839844 time to convert the images to numpy array : 0.04051017761230469 time to import caffe and check if the image exist : 0.021630048751831055 time to convert the images to numpy array : 0.03376579284667969 time to import caffe and check if the image exist : 0.01103973388671875 time to convert the images to numpy array : 0.04530763626098633 time to import caffe and check if the image exist : 0.009847402572631836 time to convert the images to numpy array : 0.046747446060180664 total time to convert the images to numpy array : 0.3083479404449463 list photo_ids error: [] list photo_ids correct : [1336209970, 1336209971, 1336209972, 1336209973, 1336209974, 1336209976, 1336209977, 1336209978, 1336209979, 1336209981, 1336209982, 1336209983, 1336209984, 1336209985, 1336209811, 1336209812, 1336209813, 1336209814, 1336209815, 1336209817, 1336209818, 1336209819, 1336209820, 1336209821, 1336209822, 1336209823, 1336209824, 1336209825, 1336210004, 1336210005, 1336210006, 1336210007, 1336210008, 1336210009, 1336210010, 1336210011, 1336210012, 1336210013, 1336210014, 1336210015, 1336210016, 1336210017, 1336210018, 1336210019, 1336210020, 1336210021, 1336210022, 1336210023, 1336210024, 1336210025, 1336210026, 1336210027, 1336210028, 1336210029, 1336209854, 1336209855, 1336209856, 1336209857, 1336209858, 1336209859, 1336209860, 1336209861, 1336209863, 1336209865, 1336209867, 1336209869, 1336209871, 1336209873, 1336209986, 1336209987, 1336209988, 1336209989, 1336209990, 1336209992, 1336209993, 1336209994, 1336209995, 1336209996, 1336209997, 1336209998, 1336209999, 1336210003, 1336209955, 1336209956, 1336209957, 1336209958, 1336209959, 1336209960, 1336209961, 1336209962, 1336209963, 1336209964, 1336209966, 1336209967, 1336209968, 1336209969, 1336209826, 1336209827, 1336209828, 1336209829, 1336209830, 1336209831, 1336209832, 1336209833, 1336209834, 1336209835, 1336209836, 1336209837, 1336209838, 1336209839, 1336209840, 1336209841, 1336209842, 1336209843, 1336209844, 1336209845, 1336209846, 1336209847, 1336209848, 1336209849, 1336209850, 1336209851, 1336209852, 1336209853, 1336209875, 1336209877, 1336209879, 1336209881, 1336209883, 1336209885, 1336209887, 1336209889, 1336209891, 1336209893, 1336209895, 1336209897, 1336209899, 1336209901] number of photos to traite : 138 try to delete the photos incorrect in DB tagging for thcl : 3237 To do loadFromThcl(), then load ParamDescType : thcl3237 thcls : [{'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'}] thcl {'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'} Update svm_hashtag_type_desc : 5561 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) To loadFromThcl() : net_5561 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10774 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_rubbia_refus_2500 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_rubbia_refus_2500 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_rubbia_refus_2500 /data/models_weight/learn_rubbia_refus_2500/caffemodel size_local : 94358479 size in s3 : 94358479 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:16 caffemodel already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt size_local : 32544 size in s3 : 32544 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:15 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:09:52 mean.npy already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/synset_words.txt size_local : 334 size in s3 : 334 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:10:06 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt caffemodel_filename : /data/models_weight/learn_rubbia_refus_2500/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10555 max_wait_temp : 1 max_wait : 0 dict_keys(['pool5', 'prob']) time used to do the prepocess of the images : 0.6771888732910156 time used to do the prediction : 0.5666921138763428 save descriptor for thcl : 3237 time to traite the descriptors : 0.7665526866912842 Catched exception ! Connect or reconnect ! storage_type for insertDescriptorsMulti : 3 To insert : 1336209970 To insert : 1336209971 To insert : 1336209972 To insert : 1336209973 To insert : 1336209974 To insert : 1336209976 To insert : 1336209977 To insert : 1336209978 To insert : 1336209979 To insert : 1336209981 To insert : 1336209982 To insert : 1336209983 To insert : 1336209984 To insert : 1336209985 To insert : 1336209811 To insert : 1336209812 To insert : 1336209813 To insert : 1336209814 To insert : 1336209815 To insert : 1336209817 To insert : 1336209818 To insert : 1336209819 To insert : 1336209820 To insert : 1336209821 To insert : 1336209822 To insert : 1336209823 To insert : 1336209824 To insert : 1336209825 To insert : 1336210004 To insert : 1336210005 To insert : 1336210006 To insert : 1336210007 To insert : 1336210008 To insert : 1336210009 To insert : 1336210010 To insert : 1336210011 To insert : 1336210012 To insert : 1336210013 To insert : 1336210014 To insert : 1336210015 To insert : 1336210016 To insert : 1336210017 To insert : 1336210018 To insert : 1336210019 To insert : 1336210020 To insert : 1336210021 To insert : 1336210022 To insert : 1336210023 To insert : 1336210024 To insert : 1336210025 To insert : 1336210026 To insert : 1336210027 To insert : 1336210028 To insert : 1336210029 To insert : 1336209854 To insert : 1336209855 To insert : 1336209856 To insert : 1336209857 To insert : 1336209858 To insert : 1336209859 To insert : 1336209860 To insert : 1336209861 To insert : 1336209863 To insert : 1336209865 To insert : 1336209867 To insert : 1336209869 To insert : 1336209871 To insert : 1336209873 To insert : 1336209986 To insert : 1336209987 To insert : 1336209988 To insert : 1336209989 To insert : 1336209990 To insert : 1336209992 To insert : 1336209993 To insert : 1336209994 To insert : 1336209995 To insert : 1336209996 To insert : 1336209997 To insert : 1336209998 To insert : 1336209999 To insert : 1336210003 To insert : 1336209955 To insert : 1336209956 To insert : 1336209957 To insert : 1336209958 To insert : 1336209959 To insert : 1336209960 To insert : 1336209961 To insert : 1336209962 To insert : 1336209963 To insert : 1336209964 To insert : 1336209966 To insert : 1336209967 To insert : 1336209968 To insert : 1336209969 To insert : 1336209826 To insert : 1336209827 To insert : 1336209828 To insert : 1336209829 To insert : 1336209830 To insert : 1336209831 To insert : 1336209832 To insert : 1336209833 To insert : 1336209834 To insert : 1336209835 To insert : 1336209836 To insert : 1336209837 To insert : 1336209838 To insert : 1336209839 To insert : 1336209840 To insert : 1336209841 To insert : 1336209842 To insert : 1336209843 To insert : 1336209844 To insert : 1336209845 To insert : 1336209846 To insert : 1336209847 To insert : 1336209848 To insert : 1336209849 To insert : 1336209850 To insert : 1336209851 To insert : 1336209852 To insert : 1336209853 To insert : 1336209875 To insert : 1336209877 To insert : 1336209879 To insert : 1336209881 To insert : 1336209883 To insert : 1336209885 To insert : 1336209887 To insert : 1336209889 To insert : 1336209891 To insert : 1336209893 To insert : 1336209895 To insert : 1336209897 To insert : 1336209899 To insert : 1336209901 time to insert the descriptors : 21.51582646369934 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos SAVE THCL : begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 209 time used for this insertion : 0.01932811737060547 save missing photos in datou_result : time spend for datou_step_exec : 27.727920055389404 time spend to save output : 0.07189488410949707 total time spend for step 6 : 27.7998149394989 step7:ventilate_hashtags_in_portfolio Sun Feb 9 16:22:09 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 20363428 get user id for portfolio 20363428 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20363428 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','flou','pet_fonce','Carton','PET_clair','metal','pet_clair','pehd','refus','PEHD','carton','Film_plastique','autre','environnement','PET_fonce','Papier','mal_croppe','Tetrapak')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20363428 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','flou','pet_fonce','Carton','PET_clair','metal','pet_clair','pehd','refus','PEHD','carton','Film_plastique','autre','environnement','PET_fonce','Papier','mal_croppe','Tetrapak')) AND mptpi.`min_score`=0.6 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20363428 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','flou','pet_fonce','Carton','PET_clair','metal','pet_clair','pehd','refus','PEHD','carton','Film_plastique','autre','environnement','PET_fonce','Papier','mal_croppe','Tetrapak')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20363428 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','flou','pet_fonce','Carton','PET_clair','metal','pet_clair','pehd','refus','PEHD','carton','Film_plastique','autre','environnement','PET_fonce','Papier','mal_croppe','Tetrapak')) AND mptpi.`min_score`=0.6 To do lien utilise dans velours : https://www.fotonower.com/velours/20377582,20377568,20377581,20377577,20377573,20377572,20377576,20377575,20377578,20377579,20377580,20377583,20377584?tags=papier,flou,pet_fonce,carton,pet_clair,metal,pehd,refus,film_plastique,autre,environnement,mal_croppe,tetrapak&datou_id_consolidate=4235&port_consolidate=20363428 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1335899451, 1335899449] Looping around the photos to save general results len do output : 1 /20363428. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2569149') ('4234', '20363428', '1335899451', None, None, None, None, None, '2569149') ('4234', None, None, None, None, None, None, None, '2569149') ('4234', '20363428', '1335899449', None, None, None, None, None, '2569149') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.012723684310913086 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.9961464405059814 time spend to save output : 0.013049125671386719 total time spend for step 7 : 3.009195566177368 step8:final Sun Feb 9 16:22:12 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1335899451: ('0.2729580356135476',), 1335899449: ('0.2729580356135476',)} new output for save of step final : {1335899451: ('0.2729580356135476',), 1335899449: ('0.2729580356135476',)} [1335899451, 1335899449] Looping around the photos to save general results len do output : 2 /1335899451.Didn't retrieve data . /1335899449.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2569149') ('4234', '20363428', '1335899451', None, None, None, None, None, '2569149') ('4234', None, None, None, None, None, None, None, '2569149') ('4234', '20363428', '1335899449', None, None, None, None, None, '2569149') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.012619972229003906 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.14596319198608398 time spend to save output : 0.012885093688964844 total time spend for step 8 : 0.15884828567504883 step9:velours_tree Sun Feb 9 16:22:12 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure list_portfolios : 20377582,20377568,20377581,20377577,20377573,20377572,20377576,20377575,20377578,20377579,20377580,20377583,20377584 photo desc type : 5561 - Retrieving photos to tag... query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20377582 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20377568 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20377581 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20377577 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20377573 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20377572 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20377576 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20377575 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20377578 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20377579 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20377580 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20377583 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20377584 ORDER BY ph.size desc - Loading descriptors... Size : 2048 len(descriptors) : 138 Compute structured hierarchical clustering... ward : AgglomerativeClustering(n_clusters=6) ward.labels_ : [1 0 3 1 3 3 3 2 3 1 1 0 1 3 3 1 3 3 1 1 1 2 0 0 3 1 3 1 3 3 1 4 1 1 1 0 1 0 1 1 3 3 3 3 3 3 0 1 3 3 1 3 3 0 3 1 0 3 3 3 1 2 1 0 3 1 1 3 3 3 1 0 1 3 2 3 3 2 2 5 0 1 1 0 3 1 0 4 4 4 4 0 3 0 2 3 3 1 3 2 5 0 0 3 3 3 0 1 1 1 1 4 4 4 1 1 1 0 3 2 1 4 1 1 1 4 1 0 1 1 0 1 1 4 1 1 3 3] Elapsed time: 0.017417192459106445 graph_id used : 77350 - Beta version, working pretty good on 11-5-16 ! https://www.fotonower.com/velours/20377582,20377568,20377581,20377577,20377573,20377572,20377576,20377575,20377578,20377579,20377580,20377583,20377584?tags=papier,flou,pet_fonce,carton,pet_clair,metal,pehd,refus,film_plastique,autre,environnement,mal_croppe,tetrapak&datou_id_consolidate=4235&port_consolidate=20363428&tree_id=77350 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : velours_tree we use saveGeneral [1335899451, 1335899449] Looping around the photos to save general results len do output : 1 /20363428Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2569149') ('4234', '20363428', '1335899451', None, None, None, None, None, '2569149') ('4234', None, None, None, None, None, None, None, '2569149') ('4234', '20363428', '1335899449', None, None, None, None, None, '2569149') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.012355566024780273 save_final save missing photos in datou_result : time spend for datou_step_exec : 8.287165880203247 time spend to save output : 0.012523174285888672 total time spend for step 9 : 8.299689054489136 step10:send_mail_cod Sun Feb 9 16:22:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Qualipapia_P20363428_09-02-2025_16_22_21.pdf 20377564 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203775641739114541 20377550 imagette203775501739114542 20377563 imagette203775631739114542 20377559 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203775591739114542 20377555 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203775551739114543 20377554 imagette203775541739114543 20377558 change filename to text .change filename to text .imagette203775581739114543 20377557 imagette203775571739114543 20377560 imagette203775601739114543 20377561 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203775611739114543 20377565 imagette203775651739114544 20377566 imagette203775661739114544 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20363428 and hashtag_type = 4230 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20377582,20377568,20377581,20377577,20377573,20377572,20377576,20377575,20377578,20377579,20377580,20377583,20377584?tags=papier,flou,pet_fonce,carton,pet_clair,metal,pehd,refus,film_plastique,autre,environnement,mal_croppe,tetrapak&datou_id_consolidate=4235&port_consolidate=20363428&tree_id=77350 your option no_mail is active, we will not send the real mail to your client args[1335899451] : ((1335899451, -7.3465638826423385, 492609224), (1335899451, -0.5812396445517285, 501862349), '0.2729580356135476') apple ((1335899451, -7.3465638826423385, 492609224), (1335899451, -0.5812396445517285, 501862349), '0.2729580356135476') We are sending mail with results at cod@fotonower.com args[1335899449] : ((1335899449, -7.343381079546548, 492609224), (1335899449, -0.6040512583120135, 501862349), '0.2729580356135476') apple ((1335899449, -7.343381079546548, 492609224), (1335899449, -0.6040512583120135, 501862349), '0.2729580356135476') We are sending mail with results at cod@fotonower.com refus_total : 0.2729580356135476 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20363428 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20363428_09-02-2025_16_22_21.pdf results_Qualipapia_P20363428_09-02-2025_16_22_21.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20363428_09-02-2025_16_22_21.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4234','20363428','results_Qualipapia_P20363428_09-02-2025_16_22_21.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20363428_09-02-2025_16_22_21.pdf','pdf','','0.15','0.2729580356135476') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1335899451, 1335899449] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2569149') ('4234', '20363428', '1335899451', None, None, None, None, None, '2569149') ('4234', None, None, None, None, None, None, None, '2569149') ('4234', '20363428', '1335899449', None, None, None, None, None, '2569149') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 2 time used for this insertion : 0.01275491714477539 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.440743684768677 time spend to save output : 0.012975454330444336 total time spend for step 10 : 4.453719139099121 step11:split_time_score Sun Feb 9 16:22:25 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 3442, 'mtr_user_id': 31, 'name': 'classifieur_2camions_valcor_021122_v1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'deux_camions,camion_droite,camion_gauche,pas_de_camion', 'svm_portfolios_learning': '7659379,7659034,7657685,7657114', 'photo_hashtag_type': 4458, 'photo_desc_type': 5723, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107760533,2107760534,2107760535,2107760536'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('15', 2),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 08022025 20363428 Nombre de photos uploadées : 2 / 23040 (0%) 08022025 20363428 Nombre de photos taguées (types de déchets): 2 / 2 (100%) 08022025 20363428 Nombre de photos taguées (volume) : 0 / 2 (0%) elapsed_time : load_data_split_time_score 2.6226043701171875e-06 elapsed_time : order_list_meta_photo_and_scores 7.3909759521484375e-06 ?? elapsed_time : fill_and_build_computed_from_old_data 0.0002143383026123047 elapsed_time : insert_dashboard_record_day_entry 0.024620771408081055 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.27818061637536023 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20362227_08-02-2025_16_34_43.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20362227 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20362227 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20362228 order by id desc limit 1 Qualite : 0.22223092092802155 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20362229_08-02-2025_16_10_00.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20362229 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20362229 AND mptpi.`type`=4230 To do Qualite : 0.25971977410426256 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20362230_08-02-2025_16_23_47.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20362230 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20362230 AND mptpi.`type`=4230 To do Qualite : 0.22386833870804138 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20362231_08-02-2025_16_24_46.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20362231 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20362231 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20362470 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20362471 order by id desc limit 1 Qualite : 0.30816875834790897 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20362472_08-02-2025_17_23_23.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20362472 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20362472 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20362473 order by id desc limit 1 Qualite : 0.2988485025853143 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20362474_08-02-2025_17_35_52.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20362474 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20362474 AND mptpi.`type`=4230 To do Qualite : 0.30213882478687565 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20362475_08-02-2025_16_47_17.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20362475 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20362475 AND mptpi.`type`=4230 To do Qualite : 0.2594528173100419 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20362476_08-02-2025_17_17_37.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20362476 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20362476 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363408 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363409 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363410 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363411 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20362961 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20362962 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20362963 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20362964 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20362965 order by id desc limit 1 Qualite : 0.31289641168099197 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20362966_08-02-2025_17_47_45.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20362966 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20362966 AND mptpi.`type`=4230 To do Qualite : 0.2899477401744621 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20362967_08-02-2025_17_35_00.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20362967 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20362967 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363412 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363413 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363414 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363415 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363416 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363417 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363418 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363419 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363420 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363422 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363424 order by id desc limit 1 Qualite : 0.27548279417565125 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20363425_09-02-2025_15_50_38.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363425 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20363425 AND mptpi.`type`=4230 To do Qualite : 0.2607602496562249 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20363426_09-02-2025_16_16_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363426 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20363426 AND mptpi.`type`=4230 To do Qualite : 0.2729580356135476 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20363428_09-02-2025_16_22_21.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363428 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20363428 AND mptpi.`type`=4230 To do Qualite : 0.2134663461394173 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20363429_09-02-2025_15_32_44.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363429 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20363429 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363430 order by id desc limit 1 Qualite : 0.23458389361290446 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20363431_08-02-2025_18_23_42.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363431 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20363431 AND mptpi.`type`=4230 To do Qualite : 0.33019615792095425 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20363432_08-02-2025_18_33_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363432 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20363432 AND mptpi.`type`=4230 To do Qualite : 0.27003850791965683 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20363782_09-02-2025_15_52_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363782 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20363782 AND mptpi.`type`=4230 To do Qualite : 0.3050723250260371 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20363783_08-02-2025_20_33_25.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363783 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20363783 AND mptpi.`type`=4230 To do Qualite : 0.284163315237253 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20363784_08-02-2025_20_58_13.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363784 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20363784 AND mptpi.`type`=4230 To do Qualite : 0.2606545044769342 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20363785_08-02-2025_19_49_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363785 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20363785 AND mptpi.`type`=4230 To do Qualite : 0.266668048335223 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20363786_08-02-2025_21_33_20.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363786 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20363786 AND mptpi.`type`=4230 To do Qualite : 0.2395584206635648 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20363787_08-02-2025_19_33_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363787 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20363787 AND mptpi.`type`=4230 To do Qualite : 0.2238459498269027 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20363788_08-02-2025_18_47_47.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20363788 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20363788 AND mptpi.`type`=4230 To do Qualite : 0.22939044842743866 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20370948_09-02-2025_06_22_35.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20370948 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20370948 AND mptpi.`type`=4230 To do Qualite : 0.25295999540880326 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20370950_09-02-2025_04_50_12.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20370950 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20370950 AND mptpi.`type`=4230 To do Qualite : 0.337843135331076 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20370951_09-02-2025_04_43_38.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20370951 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20370951 AND mptpi.`type`=4230 To do Qualite : 0.2986241900532288 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20370952_09-02-2025_04_44_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20370952 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20370952 AND mptpi.`type`=4230 To do Qualite : 0.24363048909931315 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20370953_09-02-2025_04_39_30.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20370953 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20370953 AND mptpi.`type`=4230 To do Qualite : 0.30222034476794823 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20370954_09-02-2025_04_13_08.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20370954 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20370954 AND mptpi.`type`=4230 To do Qualite : 0.25915799589180577 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20370955_09-02-2025_04_20_32.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20370955 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20370955 AND mptpi.`type`=4230 To do Qualite : 0.2558362079089241 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20370956_09-02-2025_04_18_55.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20370956 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20370956 AND mptpi.`type`=4230 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'08022025': {'nb_upload': 2, 'nb_taggue_class': 2, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1335899451, 1335899449] Looping around the photos to save general results len do output : 1 /20363428Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2569149') ('4234', '20363428', '1335899451', None, None, None, None, None, '2569149') ('4234', None, None, None, None, None, None, None, '2569149') ('4234', '20363428', '1335899449', None, None, None, None, None, '2569149') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.013153791427612305 save_final save missing photos in datou_result : time spend for datou_step_exec : 21.26259446144104 time spend to save output : 0.013369321823120117 total time spend for step 11 : 21.27596378326416 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 1 len(input) + len(total_photo_id_missing) : 2 set_done_treatment 32.63user 21.66system 2:21.36elapsed 38%CPU (0avgtext+0avgdata 2477856maxresident)k 770128inputs+11320outputs (1121major+2016823minor)pagefaults 0swaps