python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 4234 -l 5 ' -s datou_current_4234 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 940055 load datou : 4234 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load THCL from format json or kwargs add thcl : 3237 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig add pdt : 5561 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4234, datou_cur_ids : ['2579684'] with mtr_portfolio_ids : ['20479191'] and first list_photo_ids : [] new path : /proc/940055/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, brightness, blur_detection, rle_unique_nms_with_priority, crop_condition, thcl, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , BFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 2 ; length of list_pids : 2 ; length of list_args : 2 time to download the photos : 0.3008992671966553 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Wed Feb 12 21:20:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10998 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-12 21:20:32.573412: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-12 21:20:32.599133: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-12 21:20:32.600737: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f5ae8000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-12 21:20:32.600789: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-12 21:20:32.603881: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-12 21:20:32.811693: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x3fb0caa0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-12 21:20:32.811732: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-12 21:20:32.812819: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-12 21:20:32.813125: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-12 21:20:32.815545: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-12 21:20:32.817861: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-12 21:20:32.818210: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-12 21:20:32.820707: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-12 21:20:32.822068: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-12 21:20:32.826631: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-12 21:20:32.828164: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-12 21:20:32.828247: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-12 21:20:32.828987: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-12 21:20:32.829001: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-12 21:20:32.829010: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-12 21:20:32.830301: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10049 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-12 21:20:33.135285: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-12 21:20:33.135405: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-12 21:20:33.135427: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-12 21:20:33.135446: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-12 21:20:33.135465: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-12 21:20:33.135482: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-12 21:20:33.135500: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-12 21:20:33.135518: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-12 21:20:33.138230: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-12 21:20:33.140174: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-12 21:20:33.140213: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-12 21:20:33.140233: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-12 21:20:33.140250: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-12 21:20:33.140267: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-12 21:20:33.140284: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-12 21:20:33.140301: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-12 21:20:33.140318: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-12 21:20:33.142888: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-12 21:20:33.142954: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-12 21:20:33.142966: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-12 21:20:33.142975: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-12 21:20:33.144801: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10049 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-12 21:20:41.416484: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-12 21:20:41.632620: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 2 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -115.39922 max: 149.71328 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 0.0001049041748046875 nb_pixel_total : 909 time to create 1 rle with old method : 0.0017251968383789062 length of segment : 33 time for calcul the mask position with numpy : 5.602836608886719e-05 nb_pixel_total : 53 time to create 1 rle with old method : 0.00012254714965820312 length of segment : 28 time for calcul the mask position with numpy : 6.270408630371094e-05 nb_pixel_total : 744 time to create 1 rle with old method : 0.0013899803161621094 length of segment : 28 time for calcul the mask position with numpy : 4.982948303222656e-05 nb_pixel_total : 20 time to create 1 rle with old method : 9.489059448242188e-05 length of segment : 9 time for calcul the mask position with numpy : 6.890296936035156e-05 nb_pixel_total : 39 time to create 1 rle with old method : 0.00018358230590820312 length of segment : 17 time for calcul the mask position with numpy : 4.00543212890625e-05 nb_pixel_total : 1 time to create 1 rle with old method : 2.1457672119140625e-05 length of segment : 1 time for calcul the mask position with numpy : 5.7697296142578125e-05 nb_pixel_total : 71 time to create 1 rle with old method : 0.0002155303955078125 length of segment : 20 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -116.28594 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 0.0001087188720703125 nb_pixel_total : 1616 time to create 1 rle with old method : 0.003544330596923828 length of segment : 67 time for calcul the mask position with numpy : 0.0001430511474609375 nb_pixel_total : 2929 time to create 1 rle with old method : 0.005216121673583984 length of segment : 143 time for calcul the mask position with numpy : 8.7738037109375e-05 nb_pixel_total : 785 time to create 1 rle with old method : 0.0015459060668945312 length of segment : 42 time for calcul the mask position with numpy : 5.984306335449219e-05 nb_pixel_total : 464 time to create 1 rle with old method : 0.0008902549743652344 length of segment : 48 time for calcul the mask position with numpy : 4.935264587402344e-05 nb_pixel_total : 217 time to create 1 rle with old method : 0.0004642009735107422 length of segment : 19 time for calcul the mask position with numpy : 7.224082946777344e-05 nb_pixel_total : 1012 time to create 1 rle with old method : 0.0018391609191894531 length of segment : 65 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -99.27422 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 0.00019550323486328125 nb_pixel_total : 7357 time to create 1 rle with old method : 0.012437582015991211 length of segment : 132 time for calcul the mask position with numpy : 0.00030112266540527344 nb_pixel_total : 6731 time to create 1 rle with old method : 0.01368856430053711 length of segment : 122 time for calcul the mask position with numpy : 0.000118255615234375 nb_pixel_total : 4286 time to create 1 rle with old method : 0.00505375862121582 length of segment : 168 time for calcul the mask position with numpy : 4.029273986816406e-05 nb_pixel_total : 108 time to create 1 rle with old method : 0.000202178955078125 length of segment : 10 time for calcul the mask position with numpy : 5.5789947509765625e-05 nb_pixel_total : 952 time to create 1 rle with old method : 0.0012216567993164062 length of segment : 64 time for calcul the mask position with numpy : 0.0005254745483398438 nb_pixel_total : 21660 time to create 1 rle with old method : 0.025929927825927734 length of segment : 331 time for calcul the mask position with numpy : 6.890296936035156e-05 nb_pixel_total : 1390 time to create 1 rle with old method : 0.0019385814666748047 length of segment : 25 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -121.50469 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 5.3882598876953125e-05 nb_pixel_total : 468 time to create 1 rle with old method : 0.0007603168487548828 length of segment : 24 time for calcul the mask position with numpy : 6.151199340820312e-05 nb_pixel_total : 1091 time to create 1 rle with old method : 0.0014469623565673828 length of segment : 69 time for calcul the mask position with numpy : 4.410743713378906e-05 nb_pixel_total : 104 time to create 1 rle with old method : 0.00020313262939453125 length of segment : 17 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 247.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 135.79141 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 0.00012826919555664062 nb_pixel_total : 495 time to create 1 rle with old method : 0.003701925277709961 length of segment : 22 time for calcul the mask position with numpy : 0.0010912418365478516 nb_pixel_total : 45802 time to create 1 rle with old method : 0.0597686767578125 length of segment : 277 Processing 1 images image shape: (400, 400, 3) min: 16.00000 max: 236.00000 molded_images shape: (1, 640, 640, 3) min: -76.62578 max: 98.36562 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.07891 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 5.412101745605469e-05 nb_pixel_total : 1311 time to create 1 rle with old method : 0.0017461776733398438 length of segment : 28 time for calcul the mask position with numpy : 7.939338684082031e-05 nb_pixel_total : 2488 time to create 1 rle with old method : 0.0032732486724853516 length of segment : 49 time for calcul the mask position with numpy : 4.0531158447265625e-05 nb_pixel_total : 252 time to create 1 rle with old method : 0.0003554821014404297 length of segment : 17 time for calcul the mask position with numpy : 5.364418029785156e-05 nb_pixel_total : 1199 time to create 1 rle with old method : 0.001674652099609375 length of segment : 51 time for calcul the mask position with numpy : 5.245208740234375e-05 nb_pixel_total : 796 time to create 1 rle with old method : 0.0010442733764648438 length of segment : 32 time for calcul the mask position with numpy : 3.933906555175781e-05 nb_pixel_total : 336 time to create 1 rle with old method : 0.0005114078521728516 length of segment : 16 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -100.98906 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 6.818771362304688e-05 nb_pixel_total : 768 time to create 1 rle with old method : 0.001117706298828125 length of segment : 16 time for calcul the mask position with numpy : 6.818771362304688e-05 nb_pixel_total : 821 time to create 1 rle with old method : 0.0012717247009277344 length of segment : 40 time for calcul the mask position with numpy : 7.510185241699219e-05 nb_pixel_total : 815 time to create 1 rle with old method : 0.0010941028594970703 length of segment : 48 time for calcul the mask position with numpy : 6.771087646484375e-05 nb_pixel_total : 721 time to create 1 rle with old method : 0.0010766983032226562 length of segment : 34 time for calcul the mask position with numpy : 6.4849853515625e-05 nb_pixel_total : 959 time to create 1 rle with old method : 0.0013937950134277344 length of segment : 43 time for calcul the mask position with numpy : 0.0001251697540283203 nb_pixel_total : 1047 time to create 1 rle with old method : 0.0013010501861572266 length of segment : 148 time for calcul the mask position with numpy : 5.078315734863281e-05 nb_pixel_total : 133 time to create 1 rle with old method : 0.0002410411834716797 length of segment : 18 time for calcul the mask position with numpy : 5.2928924560546875e-05 nb_pixel_total : 733 time to create 1 rle with old method : 0.0009202957153320312 length of segment : 30 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -121.96953 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 0.0004291534423828125 nb_pixel_total : 17181 time to create 1 rle with old method : 0.021625041961669922 length of segment : 288 time for calcul the mask position with numpy : 9.083747863769531e-05 nb_pixel_total : 774 time to create 1 rle with old method : 0.0013723373413085938 length of segment : 36 time for calcul the mask position with numpy : 5.817413330078125e-05 nb_pixel_total : 137 time to create 1 rle with old method : 0.00030612945556640625 length of segment : 31 time for calcul the mask position with numpy : 8.058547973632812e-05 nb_pixel_total : 1339 time to create 1 rle with old method : 0.0023932456970214844 length of segment : 40 time for calcul the mask position with numpy : 4.887580871582031e-05 nb_pixel_total : 132 time to create 1 rle with old method : 0.000301361083984375 length of segment : 11 time for calcul the mask position with numpy : 6.961822509765625e-05 nb_pixel_total : 850 time to create 1 rle with old method : 0.0015606880187988281 length of segment : 38 time for calcul the mask position with numpy : 4.863739013671875e-05 nb_pixel_total : 143 time to create 1 rle with old method : 0.000316619873046875 length of segment : 11 time for calcul the mask position with numpy : 7.677078247070312e-05 nb_pixel_total : 1366 time to create 1 rle with old method : 0.0024361610412597656 length of segment : 41 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 254.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 131.31484 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 6.270408630371094e-05 nb_pixel_total : 331 time to create 1 rle with old method : 0.0006890296936035156 length of segment : 34 time for calcul the mask position with numpy : 6.914138793945312e-05 nb_pixel_total : 638 time to create 1 rle with old method : 0.0013377666473388672 length of segment : 68 time for calcul the mask position with numpy : 0.0003197193145751953 nb_pixel_total : 4207 time to create 1 rle with old method : 0.007946491241455078 length of segment : 212 time for calcul the mask position with numpy : 8.034706115722656e-05 nb_pixel_total : 1275 time to create 1 rle with old method : 0.0016515254974365234 length of segment : 68 time for calcul the mask position with numpy : 0.00019931793212890625 nb_pixel_total : 3281 time to create 1 rle with old method : 0.004015684127807617 length of segment : 174 time for calcul the mask position with numpy : 0.00020742416381835938 nb_pixel_total : 3692 time to create 1 rle with old method : 0.004914522171020508 length of segment : 187 Processing 1 images image shape: (280, 400, 3) min: 16.00000 max: 203.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 72.04922 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0015876293182373047 nb_pixel_total : 106414 time to create 1 rle with old method : 0.1657848358154297 length of segment : 280 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 149.51797 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 20 time for calcul the mask position with numpy : 0.00011444091796875 nb_pixel_total : 3615 time to create 1 rle with old method : 0.00445866584777832 length of segment : 63 time for calcul the mask position with numpy : 4.696846008300781e-05 nb_pixel_total : 367 time to create 1 rle with old method : 0.0005667209625244141 length of segment : 30 time for calcul the mask position with numpy : 0.00013017654418945312 nb_pixel_total : 3034 time to create 1 rle with old method : 0.004006624221801758 length of segment : 112 time for calcul the mask position with numpy : 4.291534423828125e-05 nb_pixel_total : 116 time to create 1 rle with old method : 0.00019621849060058594 length of segment : 12 time for calcul the mask position with numpy : 5.221366882324219e-05 nb_pixel_total : 505 time to create 1 rle with old method : 0.0007343292236328125 length of segment : 28 time for calcul the mask position with numpy : 9.775161743164062e-05 nb_pixel_total : 1384 time to create 1 rle with old method : 0.0019981861114501953 length of segment : 51 time for calcul the mask position with numpy : 5.53131103515625e-05 nb_pixel_total : 412 time to create 1 rle with old method : 0.0005784034729003906 length of segment : 25 time for calcul the mask position with numpy : 4.315376281738281e-05 nb_pixel_total : 308 time to create 1 rle with old method : 0.00045180320739746094 length of segment : 38 time for calcul the mask position with numpy : 6.437301635742188e-05 nb_pixel_total : 676 time to create 1 rle with old method : 0.001116037368774414 length of segment : 31 time for calcul the mask position with numpy : 5.14984130859375e-05 nb_pixel_total : 589 time to create 1 rle with old method : 0.0008006095886230469 length of segment : 39 time for calcul the mask position with numpy : 6.890296936035156e-05 nb_pixel_total : 740 time to create 1 rle with old method : 0.0010597705841064453 length of segment : 32 time for calcul the mask position with numpy : 3.62396240234375e-05 nb_pixel_total : 169 time to create 1 rle with old method : 0.0002551078796386719 length of segment : 20 time for calcul the mask position with numpy : 6.175041198730469e-05 nb_pixel_total : 1546 time to create 1 rle with old method : 0.001912832260131836 length of segment : 58 time for calcul the mask position with numpy : 5.936622619628906e-05 nb_pixel_total : 298 time to create 1 rle with old method : 0.0006158351898193359 length of segment : 27 time for calcul the mask position with numpy : 5.5789947509765625e-05 nb_pixel_total : 538 time to create 1 rle with old method : 0.0009353160858154297 length of segment : 43 time for calcul the mask position with numpy : 5.626678466796875e-05 nb_pixel_total : 1477 time to create 1 rle with old method : 0.0018696784973144531 length of segment : 44 time for calcul the mask position with numpy : 4.553794860839844e-05 nb_pixel_total : 498 time to create 1 rle with old method : 0.0007481575012207031 length of segment : 30 time for calcul the mask position with numpy : 5.459785461425781e-05 nb_pixel_total : 223 time to create 1 rle with old method : 0.00039076805114746094 length of segment : 13 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 432 time to create 1 rle with old method : 0.0005967617034912109 length of segment : 25 time for calcul the mask position with numpy : 4.863739013671875e-05 nb_pixel_total : 698 time to create 1 rle with old method : 0.0011391639709472656 length of segment : 32 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 22 time for calcul the mask position with numpy : 0.00010395050048828125 nb_pixel_total : 450 time to create 1 rle with old method : 0.000827789306640625 length of segment : 22 time for calcul the mask position with numpy : 7.939338684082031e-05 nb_pixel_total : 402 time to create 1 rle with old method : 0.001047372817993164 length of segment : 28 time for calcul the mask position with numpy : 8.463859558105469e-05 nb_pixel_total : 465 time to create 1 rle with old method : 0.0010323524475097656 length of segment : 27 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 322 time to create 1 rle with old method : 0.0005402565002441406 length of segment : 17 time for calcul the mask position with numpy : 4.1961669921875e-05 nb_pixel_total : 70 time to create 1 rle with old method : 0.0001456737518310547 length of segment : 6 time for calcul the mask position with numpy : 3.981590270996094e-05 nb_pixel_total : 367 time to create 1 rle with old method : 0.0005323886871337891 length of segment : 25 time for calcul the mask position with numpy : 3.4809112548828125e-05 nb_pixel_total : 182 time to create 1 rle with old method : 0.00026702880859375 length of segment : 22 time for calcul the mask position with numpy : 3.814697265625e-05 nb_pixel_total : 220 time to create 1 rle with old method : 0.00037407875061035156 length of segment : 12 time for calcul the mask position with numpy : 4.38690185546875e-05 nb_pixel_total : 145 time to create 1 rle with old method : 0.00025916099548339844 length of segment : 16 time for calcul the mask position with numpy : 4.5299530029296875e-05 nb_pixel_total : 302 time to create 1 rle with old method : 0.00047397613525390625 length of segment : 24 time for calcul the mask position with numpy : 4.696846008300781e-05 nb_pixel_total : 179 time to create 1 rle with old method : 0.00032711029052734375 length of segment : 19 time for calcul the mask position with numpy : 9.679794311523438e-05 nb_pixel_total : 180 time to create 1 rle with old method : 0.00027561187744140625 length of segment : 17 time for calcul the mask position with numpy : 4.76837158203125e-05 nb_pixel_total : 467 time to create 1 rle with old method : 0.0008013248443603516 length of segment : 24 time for calcul the mask position with numpy : 4.6253204345703125e-05 nb_pixel_total : 295 time to create 1 rle with old method : 0.0005347728729248047 length of segment : 15 time for calcul the mask position with numpy : 6.318092346191406e-05 nb_pixel_total : 558 time to create 1 rle with old method : 0.0008025169372558594 length of segment : 32 time for calcul the mask position with numpy : 8.559226989746094e-05 nb_pixel_total : 849 time to create 1 rle with old method : 0.001287698745727539 length of segment : 30 time for calcul the mask position with numpy : 4.291534423828125e-05 nb_pixel_total : 106 time to create 1 rle with old method : 0.00020694732666015625 length of segment : 9 time for calcul the mask position with numpy : 4.3392181396484375e-05 nb_pixel_total : 133 time to create 1 rle with old method : 0.00022840499877929688 length of segment : 13 time for calcul the mask position with numpy : 4.744529724121094e-05 nb_pixel_total : 405 time to create 1 rle with old method : 0.0005652904510498047 length of segment : 26 time for calcul the mask position with numpy : 6.628036499023438e-05 nb_pixel_total : 285 time to create 1 rle with old method : 0.0005996227264404297 length of segment : 20 time for calcul the mask position with numpy : 6.318092346191406e-05 nb_pixel_total : 653 time to create 1 rle with old method : 0.0008945465087890625 length of segment : 34 time for calcul the mask position with numpy : 6.461143493652344e-05 nb_pixel_total : 192 time to create 1 rle with old method : 0.00043272972106933594 length of segment : 18 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 145.13906 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 4.00543212890625e-05 nb_pixel_total : 362 time to create 1 rle with old method : 0.0005278587341308594 length of segment : 19 time for calcul the mask position with numpy : 0.00015807151794433594 nb_pixel_total : 744 time to create 1 rle with old method : 0.0010688304901123047 length of segment : 64 time for calcul the mask position with numpy : 3.695487976074219e-05 nb_pixel_total : 226 time to create 1 rle with old method : 0.0003275871276855469 length of segment : 21 time for calcul the mask position with numpy : 3.361701965332031e-05 nb_pixel_total : 263 time to create 1 rle with old method : 0.0003826618194580078 length of segment : 25 time for calcul the mask position with numpy : 3.528594970703125e-05 nb_pixel_total : 189 time to create 1 rle with old method : 0.00029850006103515625 length of segment : 21 time for calcul the mask position with numpy : 3.123283386230469e-05 nb_pixel_total : 108 time to create 1 rle with old method : 0.00017833709716796875 length of segment : 11 time for calcul the mask position with numpy : 3.62396240234375e-05 nb_pixel_total : 240 time to create 1 rle with old method : 0.0003249645233154297 length of segment : 28 time for calcul the mask position with numpy : 3.4809112548828125e-05 nb_pixel_total : 211 time to create 1 rle with old method : 0.0003077983856201172 length of segment : 24 time for calcul the mask position with numpy : 3.0517578125e-05 nb_pixel_total : 51 time to create 1 rle with old method : 0.00010824203491210938 length of segment : 8 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 141.72500 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 5.2928924560546875e-05 nb_pixel_total : 17 time to create 1 rle with old method : 8.559226989746094e-05 length of segment : 5 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -115.84063 max: 148.08828 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 6.842613220214844e-05 nb_pixel_total : 912 time to create 1 rle with old method : 0.0017514228820800781 length of segment : 35 time for calcul the mask position with numpy : 0.00019216537475585938 nb_pixel_total : 4069 time to create 1 rle with old method : 0.0070917606353759766 length of segment : 132 time for calcul the mask position with numpy : 8.20159912109375e-05 nb_pixel_total : 748 time to create 1 rle with old method : 0.0010831356048583984 length of segment : 26 time for calcul the mask position with numpy : 3.62396240234375e-05 nb_pixel_total : 67 time to create 1 rle with old method : 0.00018668174743652344 length of segment : 24 time for calcul the mask position with numpy : 3.7670135498046875e-05 nb_pixel_total : 54 time to create 1 rle with old method : 0.0001690387725830078 length of segment : 15 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.81328 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 5.245208740234375e-05 nb_pixel_total : 758 time to create 1 rle with old method : 0.0010237693786621094 length of segment : 29 time for calcul the mask position with numpy : 9.179115295410156e-05 nb_pixel_total : 2393 time to create 1 rle with old method : 0.0030879974365234375 length of segment : 53 time for calcul the mask position with numpy : 5.745887756347656e-05 nb_pixel_total : 902 time to create 1 rle with old method : 0.0012030601501464844 length of segment : 29 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -106.58672 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 0.00010514259338378906 nb_pixel_total : 1674 time to create 1 rle with old method : 0.0035598278045654297 length of segment : 24 time for calcul the mask position with numpy : 0.00015997886657714844 nb_pixel_total : 2255 time to create 1 rle with old method : 0.004386425018310547 length of segment : 75 time for calcul the mask position with numpy : 0.00020837783813476562 nb_pixel_total : 5285 time to create 1 rle with old method : 0.010832786560058594 length of segment : 93 time for calcul the mask position with numpy : 0.00011682510375976562 nb_pixel_total : 98 time to create 1 rle with old method : 0.0002903938293457031 length of segment : 9 time for calcul the mask position with numpy : 0.0003542900085449219 nb_pixel_total : 5951 time to create 1 rle with old method : 0.011404991149902344 length of segment : 134 time for calcul the mask position with numpy : 0.00019693374633789062 nb_pixel_total : 1033 time to create 1 rle with old method : 0.002523183822631836 length of segment : 28 time for calcul the mask position with numpy : 0.0001678466796875 nb_pixel_total : 1206 time to create 1 rle with old method : 0.0024271011352539062 length of segment : 98 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -121.18828 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 8.0108642578125e-05 nb_pixel_total : 496 time to create 1 rle with old method : 0.0010945796966552734 length of segment : 27 time for calcul the mask position with numpy : 8.797645568847656e-05 nb_pixel_total : 1311 time to create 1 rle with old method : 0.0022797584533691406 length of segment : 69 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 134.57266 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 5.3882598876953125e-05 nb_pixel_total : 507 time to create 1 rle with old method : 0.0009915828704833984 length of segment : 19 Processing 1 images image shape: (400, 400, 3) min: 15.00000 max: 236.00000 molded_images shape: (1, 640, 640, 3) min: -80.54766 max: 96.38906 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -118.65313 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 14 time for calcul the mask position with numpy : 6.937980651855469e-05 nb_pixel_total : 1300 time to create 1 rle with old method : 0.0017580986022949219 length of segment : 29 time for calcul the mask position with numpy : 4.482269287109375e-05 nb_pixel_total : 300 time to create 1 rle with old method : 0.0004467964172363281 length of segment : 28 time for calcul the mask position with numpy : 8.416175842285156e-05 nb_pixel_total : 2475 time to create 1 rle with old method : 0.003413677215576172 length of segment : 50 time for calcul the mask position with numpy : 5.555152893066406e-05 nb_pixel_total : 467 time to create 1 rle with old method : 0.0006897449493408203 length of segment : 37 time for calcul the mask position with numpy : 4.38690185546875e-05 nb_pixel_total : 294 time to create 1 rle with old method : 0.00047969818115234375 length of segment : 18 time for calcul the mask position with numpy : 4.38690185546875e-05 nb_pixel_total : 392 time to create 1 rle with old method : 0.0005872249603271484 length of segment : 17 time for calcul the mask position with numpy : 5.650520324707031e-05 nb_pixel_total : 936 time to create 1 rle with old method : 0.001275777816772461 length of segment : 49 time for calcul the mask position with numpy : 8.392333984375e-05 nb_pixel_total : 1856 time to create 1 rle with old method : 0.0023856163024902344 length of segment : 58 time for calcul the mask position with numpy : 6.222724914550781e-05 nb_pixel_total : 1240 time to create 1 rle with old method : 0.0016870498657226562 length of segment : 57 time for calcul the mask position with numpy : 0.00018930435180664062 nb_pixel_total : 6186 time to create 1 rle with old method : 0.007771492004394531 length of segment : 114 time for calcul the mask position with numpy : 5.555152893066406e-05 nb_pixel_total : 744 time to create 1 rle with old method : 0.0010023117065429688 length of segment : 29 time for calcul the mask position with numpy : 5.125999450683594e-05 nb_pixel_total : 800 time to create 1 rle with old method : 0.0010616779327392578 length of segment : 32 time for calcul the mask position with numpy : 5.221366882324219e-05 nb_pixel_total : 661 time to create 1 rle with old method : 0.0009682178497314453 length of segment : 40 time for calcul the mask position with numpy : 4.315376281738281e-05 nb_pixel_total : 380 time to create 1 rle with old method : 0.0006043910980224609 length of segment : 31 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -105.77031 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 5.1975250244140625e-05 nb_pixel_total : 1025 time to create 1 rle with old method : 0.0013234615325927734 length of segment : 50 time for calcul the mask position with numpy : 5.054473876953125e-05 nb_pixel_total : 560 time to create 1 rle with old method : 0.0008974075317382812 length of segment : 33 time for calcul the mask position with numpy : 6.604194641113281e-05 nb_pixel_total : 609 time to create 1 rle with old method : 0.0010080337524414062 length of segment : 34 time for calcul the mask position with numpy : 4.1961669921875e-05 nb_pixel_total : 700 time to create 1 rle with old method : 0.001043081283569336 length of segment : 16 time for calcul the mask position with numpy : 5.698204040527344e-05 nb_pixel_total : 952 time to create 1 rle with old method : 0.0012652873992919922 length of segment : 71 time for calcul the mask position with numpy : 3.504753112792969e-05 nb_pixel_total : 354 time to create 1 rle with old method : 0.0005984306335449219 length of segment : 17 time for calcul the mask position with numpy : 4.458427429199219e-05 nb_pixel_total : 712 time to create 1 rle with old method : 0.0009100437164306641 length of segment : 30 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -121.97344 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 0.0004639625549316406 nb_pixel_total : 20424 time to create 1 rle with old method : 0.02347588539123535 length of segment : 298 time for calcul the mask position with numpy : 7.152557373046875e-05 nb_pixel_total : 846 time to create 1 rle with old method : 0.0011296272277832031 length of segment : 37 time for calcul the mask position with numpy : 5.9604644775390625e-05 nb_pixel_total : 1329 time to create 1 rle with old method : 0.0017180442810058594 length of segment : 40 time for calcul the mask position with numpy : 6.4849853515625e-05 nb_pixel_total : 981 time to create 1 rle with old method : 0.001462697982788086 length of segment : 40 time for calcul the mask position with numpy : 6.341934204101562e-05 nb_pixel_total : 1368 time to create 1 rle with old method : 0.0017969608306884766 length of segment : 42 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 252.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 128.74062 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 3.981590270996094e-05 nb_pixel_total : 318 time to create 1 rle with old method : 0.0004477500915527344 length of segment : 33 time for calcul the mask position with numpy : 4.8160552978515625e-05 nb_pixel_total : 523 time to create 1 rle with old method : 0.0007846355438232422 length of segment : 53 time for calcul the mask position with numpy : 0.0001933574676513672 nb_pixel_total : 4182 time to create 1 rle with old method : 0.005015850067138672 length of segment : 208 time for calcul the mask position with numpy : 0.00021314620971679688 nb_pixel_total : 3874 time to create 1 rle with old method : 0.004660606384277344 length of segment : 193 time for calcul the mask position with numpy : 0.00022077560424804688 nb_pixel_total : 3408 time to create 1 rle with old method : 0.004535198211669922 length of segment : 180 Processing 1 images image shape: (280, 400, 3) min: 4.00000 max: 206.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 76.58828 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0010352134704589844 nb_pixel_total : 106209 time to create 1 rle with old method : 0.1233530044555664 length of segment : 281 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 17 time for calcul the mask position with numpy : 7.987022399902344e-05 nb_pixel_total : 3518 time to create 1 rle with old method : 0.004403829574584961 length of segment : 58 time for calcul the mask position with numpy : 3.981590270996094e-05 nb_pixel_total : 546 time to create 1 rle with old method : 0.0007610321044921875 length of segment : 30 time for calcul the mask position with numpy : 7.486343383789062e-05 nb_pixel_total : 1814 time to create 1 rle with old method : 0.0022444725036621094 length of segment : 93 time for calcul the mask position with numpy : 0.0001220703125 nb_pixel_total : 3136 time to create 1 rle with old method : 0.004147768020629883 length of segment : 93 time for calcul the mask position with numpy : 4.315376281738281e-05 nb_pixel_total : 324 time to create 1 rle with old method : 0.0004439353942871094 length of segment : 27 time for calcul the mask position with numpy : 4.363059997558594e-05 nb_pixel_total : 774 time to create 1 rle with old method : 0.0011012554168701172 length of segment : 32 time for calcul the mask position with numpy : 4.220008850097656e-05 nb_pixel_total : 118 time to create 1 rle with old method : 0.00024771690368652344 length of segment : 12 time for calcul the mask position with numpy : 4.9591064453125e-05 nb_pixel_total : 508 time to create 1 rle with old method : 0.0007843971252441406 length of segment : 26 time for calcul the mask position with numpy : 4.410743713378906e-05 nb_pixel_total : 520 time to create 1 rle with old method : 0.0007345676422119141 length of segment : 41 time for calcul the mask position with numpy : 4.5299530029296875e-05 nb_pixel_total : 645 time to create 1 rle with old method : 0.0009074211120605469 length of segment : 29 time for calcul the mask position with numpy : 3.838539123535156e-05 nb_pixel_total : 540 time to create 1 rle with old method : 0.0007002353668212891 length of segment : 35 time for calcul the mask position with numpy : 4.0531158447265625e-05 nb_pixel_total : 349 time to create 1 rle with old method : 0.0004868507385253906 length of segment : 31 time for calcul the mask position with numpy : 5.125999450683594e-05 nb_pixel_total : 683 time to create 1 rle with old method : 0.0010559558868408203 length of segment : 53 time for calcul the mask position with numpy : 3.886222839355469e-05 nb_pixel_total : 472 time to create 1 rle with old method : 0.000652313232421875 length of segment : 27 time for calcul the mask position with numpy : 4.029273986816406e-05 nb_pixel_total : 518 time to create 1 rle with old method : 0.0006494522094726562 length of segment : 41 time for calcul the mask position with numpy : 3.9577484130859375e-05 nb_pixel_total : 544 time to create 1 rle with old method : 0.0006990432739257812 length of segment : 42 time for calcul the mask position with numpy : 4.3392181396484375e-05 nb_pixel_total : 686 time to create 1 rle with old method : 0.0010178089141845703 length of segment : 30 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 14 time for calcul the mask position with numpy : 6.818771362304688e-05 nb_pixel_total : 348 time to create 1 rle with old method : 0.0004868507385253906 length of segment : 27 time for calcul the mask position with numpy : 4.0531158447265625e-05 nb_pixel_total : 463 time to create 1 rle with old method : 0.0006046295166015625 length of segment : 27 time for calcul the mask position with numpy : 4.601478576660156e-05 nb_pixel_total : 462 time to create 1 rle with old method : 0.0006935596466064453 length of segment : 23 time for calcul the mask position with numpy : 4.100799560546875e-05 nb_pixel_total : 384 time to create 1 rle with old method : 0.0005288124084472656 length of segment : 26 time for calcul the mask position with numpy : 4.458427429199219e-05 nb_pixel_total : 535 time to create 1 rle with old method : 0.0007827281951904297 length of segment : 38 time for calcul the mask position with numpy : 5.1021575927734375e-05 nb_pixel_total : 476 time to create 1 rle with old method : 0.0006778240203857422 length of segment : 31 time for calcul the mask position with numpy : 4.076957702636719e-05 nb_pixel_total : 208 time to create 1 rle with old method : 0.00029206275939941406 length of segment : 25 time for calcul the mask position with numpy : 3.933906555175781e-05 nb_pixel_total : 186 time to create 1 rle with old method : 0.0002892017364501953 length of segment : 20 time for calcul the mask position with numpy : 3.4332275390625e-05 nb_pixel_total : 264 time to create 1 rle with old method : 0.000392913818359375 length of segment : 12 time for calcul the mask position with numpy : 3.337860107421875e-05 nb_pixel_total : 120 time to create 1 rle with old method : 0.00027751922607421875 length of segment : 14 time for calcul the mask position with numpy : 3.719329833984375e-05 nb_pixel_total : 343 time to create 1 rle with old method : 0.0005156993865966797 length of segment : 17 time for calcul the mask position with numpy : 3.886222839355469e-05 nb_pixel_total : 183 time to create 1 rle with old method : 0.00029397010803222656 length of segment : 17 time for calcul the mask position with numpy : 3.743171691894531e-05 nb_pixel_total : 318 time to create 1 rle with old method : 0.0004467964172363281 length of segment : 21 time for calcul the mask position with numpy : 4.76837158203125e-05 nb_pixel_total : 603 time to create 1 rle with old method : 0.0008108615875244141 length of segment : 29 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 145.27969 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 5.841255187988281e-05 nb_pixel_total : 219 time to create 1 rle with old method : 0.0005340576171875 length of segment : 26 time for calcul the mask position with numpy : 5.221366882324219e-05 nb_pixel_total : 232 time to create 1 rle with old method : 0.0005004405975341797 length of segment : 23 time for calcul the mask position with numpy : 4.76837158203125e-05 nb_pixel_total : 223 time to create 1 rle with old method : 0.0003292560577392578 length of segment : 22 time for calcul the mask position with numpy : 3.886222839355469e-05 nb_pixel_total : 289 time to create 1 rle with old method : 0.0005879402160644531 length of segment : 30 time for calcul the mask position with numpy : 4.315376281738281e-05 nb_pixel_total : 63 time to create 1 rle with old method : 0.00017833709716796875 length of segment : 8 time for calcul the mask position with numpy : 0.00011157989501953125 nb_pixel_total : 361 time to create 1 rle with old method : 0.0008664131164550781 length of segment : 67 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 141.72500 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Detection mask done ! Trying to reset tf kernel 940236 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 7 tf kernel not reseted sub process len(results) : 193 len(list_Values) 193 None max_time_sub_proc : 3600 parent process len(results) : 0 len(list_Values) 193 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5857 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Catched exception ! Connect or reconnect ! Number saved : None batch 1 Loaded 193 chid ids of type : 4228 Number RLEs to save : 9672 save missing photos in datou_result : time spend for datou_step_exec : 24.186092376708984 time spend to save output : 0.839287519454956 total time spend for step 1 : 25.02537989616394 step2:brightness Wed Feb 12 21:20:54 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1739391629_940055_1337117278_f0cfaaee960bdfded1def09fcb08401f.jpg treat image : temp/1739391629_940055_1337117273_d0e66ca96060d0976f166acc96b4ef26.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 2 time used for this insertion : 0.012596845626831055 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 2 time used for this insertion : 0.01650524139404297 save missing photos in datou_result : time spend for datou_step_exec : 1.0833795070648193 time spend to save output : 0.034026384353637695 total time spend for step 2 : 1.117405891418457 step3:blur_detection Wed Feb 12 21:20:55 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1739391629_940055_1337117278_f0cfaaee960bdfded1def09fcb08401f.jpg resize: (1080, 1920) 1337117278 -7.813817636105352 treat image : temp/1739391629_940055_1337117273_d0e66ca96060d0976f166acc96b4ef26.jpg resize: (1080, 1920) 1337117273 -7.83440878842373 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 2 time used for this insertion : 0.013777732849121094 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 2 time used for this insertion : 0.013613462448120117 save missing photos in datou_result : time spend for datou_step_exec : 1.935809850692749 time spend to save output : 0.032332658767700195 total time spend for step 3 : 1.9681425094604492 step4:rle_unique_nms_with_priority Wed Feb 12 21:20:57 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 193 chid ids of type : 4228 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 106 nb_hashtags : 7 time to prepare the origin masks : 1.096813678741455 time for calcul the mask position with numpy : 0.10209369659423828 nb_pixel_total : 1807734 time to create 1 rle with new method : 0.24733948707580566 time for calcul the mask position with numpy : 0.008733034133911133 nb_pixel_total : 952 time to create 1 rle with old method : 0.0011379718780517578 time for calcul the mask position with numpy : 0.008782148361206055 nb_pixel_total : 7357 time to create 1 rle with old method : 0.008539438247680664 time for calcul the mask position with numpy : 0.00893712043762207 nb_pixel_total : 21660 time to create 1 rle with old method : 0.024581432342529297 time for calcul the mask position with numpy : 0.008482694625854492 nb_pixel_total : 2929 time to create 1 rle with old method : 0.0034475326538085938 time for calcul the mask position with numpy : 0.008428812026977539 nb_pixel_total : 53 time to create 1 rle with old method : 6.771087646484375e-05 time for calcul the mask position with numpy : 0.008388757705688477 nb_pixel_total : 785 time to create 1 rle with old method : 0.0009598731994628906 time for calcul the mask position with numpy : 0.008456230163574219 nb_pixel_total : 909 time to create 1 rle with old method : 0.0011332035064697266 time for calcul the mask position with numpy : 0.008559942245483398 nb_pixel_total : 31 time to create 1 rle with old method : 6.389617919921875e-05 time for calcul the mask position with numpy : 0.008543968200683594 nb_pixel_total : 31 time to create 1 rle with old method : 8.511543273925781e-05 time for calcul the mask position with numpy : 0.008742332458496094 nb_pixel_total : 7 time to create 1 rle with old method : 4.9591064453125e-05 time for calcul the mask position with numpy : 0.008536815643310547 nb_pixel_total : 4286 time to create 1 rle with old method : 0.007355213165283203 time for calcul the mask position with numpy : 0.008817434310913086 nb_pixel_total : 20 time to create 1 rle with old method : 4.458427429199219e-05 time for calcul the mask position with numpy : 0.008769512176513672 nb_pixel_total : 45802 time to create 1 rle with old method : 0.05139565467834473 time for calcul the mask position with numpy : 0.008472681045532227 nb_pixel_total : 1 time to create 1 rle with old method : 1.0728836059570312e-05 time for calcul the mask position with numpy : 0.008416414260864258 nb_pixel_total : 108 time to create 1 rle with old method : 0.00013947486877441406 time for calcul the mask position with numpy : 0.008409976959228516 nb_pixel_total : 622 time to create 1 rle with old method : 0.0007371902465820312 time for calcul the mask position with numpy : 0.008344650268554688 nb_pixel_total : 1616 time to create 1 rle with old method : 0.001973867416381836 time for calcul the mask position with numpy : 0.00837564468383789 nb_pixel_total : 28 time to create 1 rle with old method : 3.933906555175781e-05 time for calcul the mask position with numpy : 0.008437395095825195 nb_pixel_total : 1091 time to create 1 rle with old method : 0.0012965202331542969 time for calcul the mask position with numpy : 0.008446931838989258 nb_pixel_total : 4 time to create 1 rle with old method : 1.6689300537109375e-05 time for calcul the mask position with numpy : 0.008450508117675781 nb_pixel_total : 217 time to create 1 rle with old method : 0.0002751350402832031 time for calcul the mask position with numpy : 0.008464813232421875 nb_pixel_total : 468 time to create 1 rle with old method : 0.0005981922149658203 time for calcul the mask position with numpy : 0.00842905044555664 nb_pixel_total : 1390 time to create 1 rle with old method : 0.0017309188842773438 time for calcul the mask position with numpy : 0.008420467376708984 nb_pixel_total : 6731 time to create 1 rle with old method : 0.008063316345214844 time for calcul the mask position with numpy : 0.00852060317993164 nb_pixel_total : 495 time to create 1 rle with old method : 0.0006618499755859375 time for calcul the mask position with numpy : 0.008602619171142578 nb_pixel_total : 733 time to create 1 rle with old method : 0.0008473396301269531 time for calcul the mask position with numpy : 0.009044885635375977 nb_pixel_total : 1047 time to create 1 rle with old method : 0.0012981891632080078 time for calcul the mask position with numpy : 0.00853729248046875 nb_pixel_total : 17181 time to create 1 rle with old method : 0.021501779556274414 time for calcul the mask position with numpy : 0.009473800659179688 nb_pixel_total : 774 time to create 1 rle with old method : 0.001094818115234375 time for calcul the mask position with numpy : 0.009821414947509766 nb_pixel_total : 107 time to create 1 rle with old method : 0.0001742839813232422 time for calcul the mask position with numpy : 0.009234905242919922 nb_pixel_total : 768 time to create 1 rle with old method : 0.0009822845458984375 time for calcul the mask position with numpy : 0.009296178817749023 nb_pixel_total : 49 time to create 1 rle with old method : 0.00011992454528808594 time for calcul the mask position with numpy : 0.00900125503540039 nb_pixel_total : 1339 time to create 1 rle with old method : 0.0016589164733886719 time for calcul the mask position with numpy : 0.008766412734985352 nb_pixel_total : 796 time to create 1 rle with old method : 0.0009496212005615234 time for calcul the mask position with numpy : 0.008671045303344727 nb_pixel_total : 137 time to create 1 rle with old method : 0.00022673606872558594 time for calcul the mask position with numpy : 0.008636951446533203 nb_pixel_total : 638 time to create 1 rle with old method : 0.0008223056793212891 time for calcul the mask position with numpy : 0.008720159530639648 nb_pixel_total : 721 time to create 1 rle with old method : 0.0009267330169677734 time for calcul the mask position with numpy : 0.0085906982421875 nb_pixel_total : 4207 time to create 1 rle with old method : 0.004931449890136719 time for calcul the mask position with numpy : 0.008579254150390625 nb_pixel_total : 108 time to create 1 rle with old method : 0.000247955322265625 time for calcul the mask position with numpy : 0.008590936660766602 nb_pixel_total : 208 time to create 1 rle with old method : 0.0003719329833984375 time for calcul the mask position with numpy : 0.008559226989746094 nb_pixel_total : 815 time to create 1 rle with old method : 0.00103759765625 time for calcul the mask position with numpy : 0.008548498153686523 nb_pixel_total : 821 time to create 1 rle with old method : 0.0010437965393066406 time for calcul the mask position with numpy : 0.008523225784301758 nb_pixel_total : 171 time to create 1 rle with old method : 0.0003056526184082031 time for calcul the mask position with numpy : 0.008472442626953125 nb_pixel_total : 336 time to create 1 rle with old method : 0.0004999637603759766 time for calcul the mask position with numpy : 0.008641481399536133 nb_pixel_total : 133 time to create 1 rle with old method : 0.0002110004425048828 time for calcul the mask position with numpy : 0.008644580841064453 nb_pixel_total : 2488 time to create 1 rle with old method : 0.0030524730682373047 time for calcul the mask position with numpy : 0.008520364761352539 nb_pixel_total : 331 time to create 1 rle with old method : 0.00042700767517089844 time for calcul the mask position with numpy : 0.008581399917602539 nb_pixel_total : 132 time to create 1 rle with old method : 0.00019431114196777344 time for calcul the mask position with numpy : 0.00859832763671875 nb_pixel_total : 143 time to create 1 rle with old method : 0.0003426074981689453 time for calcul the mask position with numpy : 0.009707927703857422 nb_pixel_total : 1275 time to create 1 rle with old method : 0.001558065414428711 time for calcul the mask position with numpy : 0.008526325225830078 nb_pixel_total : 1085 time to create 1 rle with old method : 0.0013892650604248047 time for calcul the mask position with numpy : 0.008516550064086914 nb_pixel_total : 252 time to create 1 rle with old method : 0.0003936290740966797 time for calcul the mask position with numpy : 0.008617401123046875 nb_pixel_total : 1311 time to create 1 rle with old method : 0.0016531944274902344 time for calcul the mask position with numpy : 0.009314775466918945 nb_pixel_total : 106414 time to create 1 rle with old method : 0.11766743659973145 time for calcul the mask position with numpy : 0.008737564086914062 nb_pixel_total : 3615 time to create 1 rle with old method : 0.004299163818359375 time for calcul the mask position with numpy : 0.008714914321899414 nb_pixel_total : 1546 time to create 1 rle with old method : 0.0019028186798095703 time for calcul the mask position with numpy : 0.008760452270507812 nb_pixel_total : 116 time to create 1 rle with old method : 0.00016307830810546875 time for calcul the mask position with numpy : 0.008725881576538086 nb_pixel_total : 223 time to create 1 rle with old method : 0.00029754638671875 time for calcul the mask position with numpy : 0.008677959442138672 nb_pixel_total : 322 time to create 1 rle with old method : 0.00043892860412597656 time for calcul the mask position with numpy : 0.008550882339477539 nb_pixel_total : 240 time to create 1 rle with old method : 0.00038313865661621094 time for calcul the mask position with numpy : 0.008646011352539062 nb_pixel_total : 180 time to create 1 rle with old method : 0.0002586841583251953 time for calcul the mask position with numpy : 0.008705854415893555 nb_pixel_total : 182 time to create 1 rle with old method : 0.0002589225769042969 time for calcul the mask position with numpy : 0.008698463439941406 nb_pixel_total : 575 time to create 1 rle with old method : 0.0007238388061523438 time for calcul the mask position with numpy : 0.008670330047607422 nb_pixel_total : 3034 time to create 1 rle with old method : 0.0036940574645996094 time for calcul the mask position with numpy : 0.008717536926269531 nb_pixel_total : 620 time to create 1 rle with old method : 0.0007903575897216797 time for calcul the mask position with numpy : 0.008588790893554688 nb_pixel_total : 3 time to create 1 rle with old method : 6.365776062011719e-05 time for calcul the mask position with numpy : 0.008646011352539062 nb_pixel_total : 505 time to create 1 rle with old method : 0.0006313323974609375 time for calcul the mask position with numpy : 0.008610963821411133 nb_pixel_total : 467 time to create 1 rle with old method : 0.0006017684936523438 time for calcul the mask position with numpy : 0.008614301681518555 nb_pixel_total : 298 time to create 1 rle with old method : 0.0003998279571533203 time for calcul the mask position with numpy : 0.008662223815917969 nb_pixel_total : 405 time to create 1 rle with old method : 0.000507354736328125 time for calcul the mask position with numpy : 0.008732795715332031 nb_pixel_total : 465 time to create 1 rle with old method : 0.0006208419799804688 time for calcul the mask position with numpy : 0.008739471435546875 nb_pixel_total : 106 time to create 1 rle with old method : 0.000164031982421875 time for calcul the mask position with numpy : 0.008613109588623047 nb_pixel_total : 178 time to create 1 rle with old method : 0.00029158592224121094 time for calcul the mask position with numpy : 0.008656978607177734 nb_pixel_total : 1075 time to create 1 rle with old method : 0.001356363296508789 time for calcul the mask position with numpy : 0.008688211441040039 nb_pixel_total : 145 time to create 1 rle with old method : 0.00023245811462402344 time for calcul the mask position with numpy : 0.00872492790222168 nb_pixel_total : 133 time to create 1 rle with old method : 0.00018644332885742188 time for calcul the mask position with numpy : 0.008567333221435547 nb_pixel_total : 367 time to create 1 rle with old method : 0.0004665851593017578 time for calcul the mask position with numpy : 0.008634567260742188 nb_pixel_total : 108 time to create 1 rle with old method : 0.00018358230590820312 time for calcul the mask position with numpy : 0.008488893508911133 nb_pixel_total : 285 time to create 1 rle with old method : 0.0003581047058105469 time for calcul the mask position with numpy : 0.008564949035644531 nb_pixel_total : 41 time to create 1 rle with old method : 0.00013399124145507812 time for calcul the mask position with numpy : 0.009595632553100586 nb_pixel_total : 295 time to create 1 rle with old method : 0.0003783702850341797 time for calcul the mask position with numpy : 0.00847482681274414 nb_pixel_total : 169 time to create 1 rle with old method : 0.00023603439331054688 time for calcul the mask position with numpy : 0.008544683456420898 nb_pixel_total : 676 time to create 1 rle with old method : 0.0008456707000732422 time for calcul the mask position with numpy : 0.008551597595214844 nb_pixel_total : 362 time to create 1 rle with old method : 0.00045943260192871094 time for calcul the mask position with numpy : 0.008552789688110352 nb_pixel_total : 402 time to create 1 rle with old method : 0.0005183219909667969 time for calcul the mask position with numpy : 0.00853276252746582 nb_pixel_total : 70 time to create 1 rle with old method : 0.00011038780212402344 time for calcul the mask position with numpy : 0.008601903915405273 nb_pixel_total : 192 time to create 1 rle with old method : 0.00025773048400878906 time for calcul the mask position with numpy : 0.008664846420288086 nb_pixel_total : 744 time to create 1 rle with old method : 0.0010142326354980469 time for calcul the mask position with numpy : 0.008732795715332031 nb_pixel_total : 262 time to create 1 rle with old method : 0.0003795623779296875 time for calcul the mask position with numpy : 0.00864100456237793 nb_pixel_total : 7 time to create 1 rle with old method : 3.8623809814453125e-05 time for calcul the mask position with numpy : 0.008774518966674805 nb_pixel_total : 308 time to create 1 rle with old method : 0.0004165172576904297 time for calcul the mask position with numpy : 0.008595705032348633 nb_pixel_total : 558 time to create 1 rle with old method : 0.0007054805755615234 time for calcul the mask position with numpy : 0.008634328842163086 nb_pixel_total : 17 time to create 1 rle with old method : 5.054473876953125e-05 time for calcul the mask position with numpy : 0.008689403533935547 nb_pixel_total : 226 time to create 1 rle with old method : 0.0002841949462890625 time for calcul the mask position with numpy : 0.008655071258544922 nb_pixel_total : 367 time to create 1 rle with old method : 0.0005013942718505859 time for calcul the mask position with numpy : 0.008682727813720703 nb_pixel_total : 302 time to create 1 rle with old method : 0.0003905296325683594 time for calcul the mask position with numpy : 0.00858616828918457 nb_pixel_total : 450 time to create 1 rle with old method : 0.000598907470703125 time for calcul the mask position with numpy : 0.008629322052001953 nb_pixel_total : 21 time to create 1 rle with old method : 7.05718994140625e-05 time for calcul the mask position with numpy : 0.00869297981262207 nb_pixel_total : 412 time to create 1 rle with old method : 0.0005724430084228516 time for calcul the mask position with numpy : 0.008643627166748047 nb_pixel_total : 538 time to create 1 rle with old method : 0.0008935928344726562 time for calcul the mask position with numpy : 0.009838104248046875 nb_pixel_total : 179 time to create 1 rle with old method : 0.0003407001495361328 time for calcul the mask position with numpy : 0.010817289352416992 nb_pixel_total : 653 time to create 1 rle with old method : 0.0010612010955810547 time for calcul the mask position with numpy : 0.01013636589050293 nb_pixel_total : 51 time to create 1 rle with old method : 0.00012111663818359375 time for calcul the mask position with numpy : 0.010972976684570312 nb_pixel_total : 189 time to create 1 rle with old method : 0.0004425048828125 time for calcul the mask position with numpy : 0.011200666427612305 nb_pixel_total : 849 time to create 1 rle with old method : 0.001619577407836914 time for calcul the mask position with numpy : 0.011302947998046875 nb_pixel_total : 220 time to create 1 rle with old method : 0.0007627010345458984 create new chi : 1.6104466915130615 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.00760197639465332 batch 1 Loaded 107 chid ids of type : 4230 Number RLEs to save : 10522 TO DO : save crop sub photo not yet done ! save time : 0.7115440368652344 nb_obj : 87 nb_hashtags : 8 time to prepare the origin masks : 2.028879165649414 time for calcul the mask position with numpy : 0.04154706001281738 nb_pixel_total : 1870545 time to create 1 rle with new method : 0.07142925262451172 time for calcul the mask position with numpy : 0.007448911666870117 nb_pixel_total : 1206 time to create 1 rle with old method : 0.0015473365783691406 time for calcul the mask position with numpy : 0.007541179656982422 nb_pixel_total : 5285 time to create 1 rle with old method : 0.008032560348510742 time for calcul the mask position with numpy : 0.007703304290771484 nb_pixel_total : 758 time to create 1 rle with old method : 0.0010616779327392578 time for calcul the mask position with numpy : 0.007891178131103516 nb_pixel_total : 912 time to create 1 rle with old method : 0.0012404918670654297 time for calcul the mask position with numpy : 0.007718801498413086 nb_pixel_total : 20 time to create 1 rle with old method : 8.320808410644531e-05 time for calcul the mask position with numpy : 0.0076329708099365234 nb_pixel_total : 67 time to create 1 rle with old method : 0.00011348724365234375 time for calcul the mask position with numpy : 0.007615327835083008 nb_pixel_total : 54 time to create 1 rle with old method : 0.00012087821960449219 time for calcul the mask position with numpy : 0.007961750030517578 nb_pixel_total : 2393 time to create 1 rle with old method : 0.003287792205810547 time for calcul the mask position with numpy : 0.00771784782409668 nb_pixel_total : 98 time to create 1 rle with old method : 0.00015926361083984375 time for calcul the mask position with numpy : 0.00772857666015625 nb_pixel_total : 4069 time to create 1 rle with old method : 0.005377054214477539 time for calcul the mask position with numpy : 0.007856369018554688 nb_pixel_total : 1311 time to create 1 rle with old method : 0.0017330646514892578 time for calcul the mask position with numpy : 0.0073239803314208984 nb_pixel_total : 496 time to create 1 rle with old method : 0.0007264614105224609 time for calcul the mask position with numpy : 0.0077056884765625 nb_pixel_total : 116 time to create 1 rle with old method : 0.0002753734588623047 time for calcul the mask position with numpy : 0.007364034652709961 nb_pixel_total : 1674 time to create 1 rle with old method : 0.002275705337524414 time for calcul the mask position with numpy : 0.007456541061401367 nb_pixel_total : 5951 time to create 1 rle with old method : 0.01073312759399414 time for calcul the mask position with numpy : 0.00896596908569336 nb_pixel_total : 2255 time to create 1 rle with old method : 0.004106998443603516 time for calcul the mask position with numpy : 0.008142709732055664 nb_pixel_total : 902 time to create 1 rle with old method : 0.0016241073608398438 time for calcul the mask position with numpy : 0.007550239562988281 nb_pixel_total : 507 time to create 1 rle with old method : 0.0009396076202392578 time for calcul the mask position with numpy : 0.00751805305480957 nb_pixel_total : 467 time to create 1 rle with old method : 0.0009136199951171875 time for calcul the mask position with numpy : 0.007645606994628906 nb_pixel_total : 354 time to create 1 rle with old method : 0.0006804466247558594 time for calcul the mask position with numpy : 0.013700246810913086 nb_pixel_total : 81 time to create 1 rle with old method : 0.0001800060272216797 time for calcul the mask position with numpy : 0.013398885726928711 nb_pixel_total : 952 time to create 1 rle with old method : 0.0015964508056640625 time for calcul the mask position with numpy : 0.010901451110839844 nb_pixel_total : 300 time to create 1 rle with old method : 0.00038433074951171875 time for calcul the mask position with numpy : 0.00623321533203125 nb_pixel_total : 712 time to create 1 rle with old method : 0.0008146762847900391 time for calcul the mask position with numpy : 0.006511688232421875 nb_pixel_total : 20424 time to create 1 rle with old method : 0.026578187942504883 time for calcul the mask position with numpy : 0.0077130794525146484 nb_pixel_total : 846 time to create 1 rle with old method : 0.0011949539184570312 time for calcul the mask position with numpy : 0.0075037479400634766 nb_pixel_total : 700 time to create 1 rle with old method : 0.0010986328125 time for calcul the mask position with numpy : 0.007544040679931641 nb_pixel_total : 71 time to create 1 rle with old method : 0.0001289844512939453 time for calcul the mask position with numpy : 0.0076045989990234375 nb_pixel_total : 1329 time to create 1 rle with old method : 0.001953125 time for calcul the mask position with numpy : 0.00781702995300293 nb_pixel_total : 735 time to create 1 rle with old method : 0.0010852813720703125 time for calcul the mask position with numpy : 0.006543874740600586 nb_pixel_total : 4321 time to create 1 rle with old method : 0.00535130500793457 time for calcul the mask position with numpy : 0.00775456428527832 nb_pixel_total : 72 time to create 1 rle with old method : 0.00010967254638671875 time for calcul the mask position with numpy : 0.007490634918212891 nb_pixel_total : 523 time to create 1 rle with old method : 0.0007851123809814453 time for calcul the mask position with numpy : 0.008350133895874023 nb_pixel_total : 392 time to create 1 rle with old method : 0.0005087852478027344 time for calcul the mask position with numpy : 0.01069498062133789 nb_pixel_total : 1856 time to create 1 rle with old method : 0.0026853084564208984 time for calcul the mask position with numpy : 0.011015653610229492 nb_pixel_total : 296 time to create 1 rle with old method : 0.0005164146423339844 time for calcul the mask position with numpy : 0.01301884651184082 nb_pixel_total : 4182 time to create 1 rle with old method : 0.005949735641479492 time for calcul the mask position with numpy : 0.013391733169555664 nb_pixel_total : 454 time to create 1 rle with old method : 0.0006542205810546875 time for calcul the mask position with numpy : 0.00822138786315918 nb_pixel_total : 41 time to create 1 rle with old method : 0.00013518333435058594 time for calcul the mask position with numpy : 0.008008956909179688 nb_pixel_total : 1025 time to create 1 rle with old method : 0.0015532970428466797 time for calcul the mask position with numpy : 0.00806427001953125 nb_pixel_total : 560 time to create 1 rle with old method : 0.0009455680847167969 time for calcul the mask position with numpy : 0.008224248886108398 nb_pixel_total : 166 time to create 1 rle with old method : 0.00035572052001953125 time for calcul the mask position with numpy : 0.00788736343383789 nb_pixel_total : 2475 time to create 1 rle with old method : 0.0047931671142578125 time for calcul the mask position with numpy : 0.00909876823425293 nb_pixel_total : 981 time to create 1 rle with old method : 0.0014514923095703125 time for calcul the mask position with numpy : 0.00784754753112793 nb_pixel_total : 318 time to create 1 rle with old method : 0.0004730224609375 time for calcul the mask position with numpy : 0.008021116256713867 nb_pixel_total : 1240 time to create 1 rle with old method : 0.0018870830535888672 time for calcul the mask position with numpy : 0.008390665054321289 nb_pixel_total : 294 time to create 1 rle with old method : 0.0004496574401855469 time for calcul the mask position with numpy : 0.008899450302124023 nb_pixel_total : 840 time to create 1 rle with old method : 0.0012600421905517578 time for calcul the mask position with numpy : 0.008363008499145508 nb_pixel_total : 1300 time to create 1 rle with old method : 0.0017518997192382812 time for calcul the mask position with numpy : 0.008530616760253906 nb_pixel_total : 106209 time to create 1 rle with old method : 0.15094923973083496 time for calcul the mask position with numpy : 0.008137702941894531 nb_pixel_total : 1785 time to create 1 rle with old method : 0.0026369094848632812 time for calcul the mask position with numpy : 0.0077402591705322266 nb_pixel_total : 3518 time to create 1 rle with old method : 0.0049664974212646484 time for calcul the mask position with numpy : 0.008033275604248047 nb_pixel_total : 118 time to create 1 rle with old method : 0.00018143653869628906 time for calcul the mask position with numpy : 0.007945775985717773 nb_pixel_total : 120 time to create 1 rle with old method : 0.00019049644470214844 time for calcul the mask position with numpy : 0.008022785186767578 nb_pixel_total : 343 time to create 1 rle with old method : 0.0005991458892822266 time for calcul the mask position with numpy : 0.007806539535522461 nb_pixel_total : 183 time to create 1 rle with old method : 0.00029349327087402344 time for calcul the mask position with numpy : 0.0076406002044677734 nb_pixel_total : 208 time to create 1 rle with old method : 0.0003132820129394531 time for calcul the mask position with numpy : 0.007738828659057617 nb_pixel_total : 472 time to create 1 rle with old method : 0.0006582736968994141 time for calcul the mask position with numpy : 0.0077893733978271484 nb_pixel_total : 289 time to create 1 rle with old method : 0.0004363059997558594 time for calcul the mask position with numpy : 0.007726430892944336 nb_pixel_total : 539 time to create 1 rle with old method : 0.0008084774017333984 time for calcul the mask position with numpy : 0.007956504821777344 nb_pixel_total : 743 time to create 1 rle with old method : 0.0010807514190673828 time for calcul the mask position with numpy : 0.007805824279785156 nb_pixel_total : 546 time to create 1 rle with old method : 0.0008950233459472656 time for calcul the mask position with numpy : 0.008039474487304688 nb_pixel_total : 3135 time to create 1 rle with old method : 0.00464177131652832 time for calcul the mask position with numpy : 0.0119781494140625 nb_pixel_total : 324 time to create 1 rle with old method : 0.0005385875701904297 time for calcul the mask position with numpy : 0.009006023406982422 nb_pixel_total : 463 time to create 1 rle with old method : 0.0007047653198242188 time for calcul the mask position with numpy : 0.011432647705078125 nb_pixel_total : 374 time to create 1 rle with old method : 0.0008080005645751953 time for calcul the mask position with numpy : 0.012900829315185547 nb_pixel_total : 349 time to create 1 rle with old method : 0.0006461143493652344 time for calcul the mask position with numpy : 0.011662960052490234 nb_pixel_total : 318 time to create 1 rle with old method : 0.0004076957702636719 time for calcul the mask position with numpy : 0.01002645492553711 nb_pixel_total : 41 time to create 1 rle with old method : 0.00010776519775390625 time for calcul the mask position with numpy : 0.01001429557800293 nb_pixel_total : 645 time to create 1 rle with old method : 0.0008029937744140625 time for calcul the mask position with numpy : 0.009065389633178711 nb_pixel_total : 348 time to create 1 rle with old method : 0.0004329681396484375 time for calcul the mask position with numpy : 0.010522842407226562 nb_pixel_total : 361 time to create 1 rle with old method : 0.0004878044128417969 time for calcul the mask position with numpy : 0.0067365169525146484 nb_pixel_total : 219 time to create 1 rle with old method : 0.0002841949462890625 time for calcul the mask position with numpy : 0.0064296722412109375 nb_pixel_total : 520 time to create 1 rle with old method : 0.0006866455078125 time for calcul the mask position with numpy : 0.0069849491119384766 nb_pixel_total : 50 time to create 1 rle with old method : 9.202957153320312e-05 time for calcul the mask position with numpy : 0.006787538528442383 nb_pixel_total : 603 time to create 1 rle with old method : 0.0007460117340087891 time for calcul the mask position with numpy : 0.007639408111572266 nb_pixel_total : 384 time to create 1 rle with old method : 0.0005598068237304688 time for calcul the mask position with numpy : 0.008028745651245117 nb_pixel_total : 232 time to create 1 rle with old method : 0.00035762786865234375 time for calcul the mask position with numpy : 0.00787353515625 nb_pixel_total : 535 time to create 1 rle with old method : 0.0008759498596191406 time for calcul the mask position with numpy : 0.008489847183227539 nb_pixel_total : 462 time to create 1 rle with old method : 0.0007565021514892578 time for calcul the mask position with numpy : 0.008543252944946289 nb_pixel_total : 508 time to create 1 rle with old method : 0.0007395744323730469 time for calcul the mask position with numpy : 0.008481264114379883 nb_pixel_total : 518 time to create 1 rle with old method : 0.0007781982421875 time for calcul the mask position with numpy : 0.008457660675048828 nb_pixel_total : 186 time to create 1 rle with old method : 0.00028204917907714844 time for calcul the mask position with numpy : 0.008169889450073242 nb_pixel_total : 476 time to create 1 rle with old method : 0.0007696151733398438 time for calcul the mask position with numpy : 0.0081634521484375 nb_pixel_total : 223 time to create 1 rle with old method : 0.0004222393035888672 time for calcul the mask position with numpy : 0.00719761848449707 nb_pixel_total : 63 time to create 1 rle with old method : 9.989738464355469e-05 time for calcul the mask position with numpy : 0.006214618682861328 nb_pixel_total : 264 time to create 1 rle with old method : 0.0003380775451660156 create new chi : 1.1439192295074463 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.0011434555053710938 batch 1 Loaded 88 chid ids of type : 4230 Number RLEs to save : 8845 TO DO : save crop sub photo not yet done ! save time : 0.5906844139099121 map_output_result : {1337117278: (0.0, 'Should be the crop_list due to order', 0.0), 1337117273: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1337117278, 1337117273] Looping around the photos to save general results len do output : 2 /1337117278.Didn't retrieve data . /1337117273.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2579684') ('4234', '20479191', '1337117278', None, None, None, None, None, '2579684') ('4234', None, None, None, None, None, None, None, '2579684') ('4234', '20479191', '1337117273', None, None, None, None, None, '2579684') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.014036178588867188 save_final save missing photos in datou_result : time spend for datou_step_exec : 8.321691989898682 time spend to save output : 0.014400482177734375 total time spend for step 4 : 8.336092472076416 step5:crop_condition Wed Feb 12 21:21:06 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4230 Loading chi in step crop for list_pids : 2 ! batch 1 Loaded 195 chid ids of type : 4230 begin to crop the class : papier param for this class : {'min_score': 0.6} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 77 About to insert : list_path_to_insert length 77 new photo from crops ! About to upload 77 photos upload in portfolio : 4869462 init cache_photo without model_param we have 77 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739391668_940055 we have uploaded 77 photos in the portfolio 4869462 time of upload the photos Elapsed time : 16.507654190063477 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.6} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 20 About to insert : list_path_to_insert length 20 new photo from crops ! About to upload 20 photos upload in portfolio : 4869462 init cache_photo without model_param we have 20 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739391685_940055 we have uploaded 20 photos in the portfolio 4869462 time of upload the photos Elapsed time : 5.690857648849487 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.6} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.6} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739391690_940055 we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.0834710597991943 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.6} filtre for class : autre hashtag_id of this class : 494826614 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 17 About to insert : list_path_to_insert length 17 new photo from crops ! About to upload 17 photos upload in portfolio : 4869462 init cache_photo without model_param we have 17 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739391692_940055 we have uploaded 17 photos in the portfolio 4869462 time of upload the photos Elapsed time : 4.141616106033325 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.6} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.6} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1337117278, 1337117273] Looping around the photos to save general results len do output : 118 /1337185842Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185844Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185846Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185848Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185850Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185852Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185854Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185856Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185858Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185860Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185862Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185864Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185866Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185868Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185870Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185872Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185874Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185876Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185878Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185880Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185882Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185884Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185886Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185888Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185890Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185892Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185895Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185898Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185900Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185902Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185904Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185906Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185908Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185910Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185912Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185914Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185916Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185918Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185920Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185922Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185924Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185926Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185928Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185930Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185932Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185934Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185936Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185938Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185940Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185942Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185944Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185946Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185948Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185950Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185952Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185954Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185956Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185958Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185960Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185962Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185964Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185966Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185968Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185970Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185972Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185974Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185976Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185978Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185980Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185982Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185984Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185986Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185988Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185990Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185992Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185994Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337185996Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186093Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186095Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186097Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186099Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186101Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186103Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186105Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186108Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186111Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186115Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186119Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186121Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186123Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186125Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186127Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186129Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186131Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186133Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186135Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186137Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186159Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186161Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186163Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186165Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186235Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186237Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186239Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186241Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186243Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186245Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186247Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186249Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186251Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186253Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186255Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186257Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186259Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186261Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186263Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186265Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337186267Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2579684') ('4234', '20479191', '1337117278', None, None, None, None, None, '2579684') ('4234', None, None, None, None, None, None, None, '2579684') ('4234', '20479191', '1337117273', None, None, None, None, None, '2579684') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 356 time used for this insertion : 0.032998085021972656 save_final save missing photos in datou_result : time spend for datou_step_exec : 30.02196979522705 time spend to save output : 0.03732728958129883 total time spend for step 5 : 30.05929708480835 step6:thcl Wed Feb 12 21:21:36 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! we are using the classfication for only one thcl 3237 time to import caffe and check if the image exist : 0.005932807922363281 time to convert the images to numpy array : 0.024442195892333984 time to import caffe and check if the image exist : 0.007965803146362305 time to convert the images to numpy array : 0.025226593017578125 time to import caffe and check if the image exist : 0.008774042129516602 time to convert the images to numpy array : 0.023419857025146484 time to import caffe and check if the image exist : 0.009360074996948242 time to convert the images to numpy array : 0.02811264991760254 time to import caffe and check if the image exist : 0.008282899856567383 time to convert the images to numpy array : 0.029998302459716797 time to import caffe and check if the image exist : 0.010281562805175781 time to convert the images to numpy array : 0.02704644203186035 time to import caffe and check if the image exist : 0.008283615112304688 time to convert the images to numpy array : 0.030165433883666992 time to import caffe and check if the image exist : 0.016274690628051758 time to convert the images to numpy array : 0.02286839485168457 time to import caffe and check if the image exist : 0.008228063583374023 time to convert the images to numpy array : 0.03154635429382324 time to import caffe and check if the image exist : 0.014472007751464844 time to convert the images to numpy array : 0.025537967681884766 total time to convert the images to numpy array : 0.2675461769104004 list photo_ids error: [] list photo_ids correct : [1337185964, 1337185966, 1337185968, 1337185970, 1337185972, 1337185974, 1337185976, 1337185978, 1337185980, 1337185982, 1337185984, 1337185986, 1337185842, 1337185844, 1337185846, 1337185848, 1337185850, 1337185852, 1337185854, 1337185856, 1337185858, 1337185860, 1337185862, 1337185864, 1337186249, 1337186251, 1337186253, 1337186255, 1337186257, 1337186259, 1337186261, 1337186263, 1337186265, 1337186267, 1337185916, 1337185918, 1337185920, 1337185922, 1337185924, 1337185926, 1337185928, 1337185930, 1337185932, 1337185934, 1337185936, 1337185938, 1337185890, 1337185892, 1337185895, 1337185898, 1337185900, 1337185902, 1337185904, 1337185906, 1337185908, 1337185910, 1337185912, 1337185914, 1337186137, 1337186159, 1337186161, 1337186163, 1337186165, 1337186235, 1337186237, 1337186239, 1337186241, 1337186243, 1337186245, 1337186247, 1337186108, 1337186111, 1337186115, 1337186119, 1337186121, 1337186123, 1337186125, 1337186127, 1337186129, 1337186131, 1337186133, 1337186135, 1337185940, 1337185942, 1337185944, 1337185946, 1337185948, 1337185950, 1337185952, 1337185954, 1337185956, 1337185958, 1337185960, 1337185962, 1337185866, 1337185868, 1337185870, 1337185872, 1337185874, 1337185876, 1337185878, 1337185880, 1337185882, 1337185884, 1337185886, 1337185888, 1337185988, 1337185990, 1337185992, 1337185994, 1337185996, 1337186093, 1337186095, 1337186097, 1337186099, 1337186101, 1337186103, 1337186105] number of photos to traite : 118 try to delete the photos incorrect in DB tagging for thcl : 3237 To do loadFromThcl(), then load ParamDescType : thcl3237 thcls : [{'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'}] thcl {'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'} Update svm_hashtag_type_desc : 5561 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) To loadFromThcl() : net_5561 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10774 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_rubbia_refus_2500 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_rubbia_refus_2500 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_rubbia_refus_2500 /data/models_weight/learn_rubbia_refus_2500/caffemodel size_local : 94358479 size in s3 : 94358479 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:16 caffemodel already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt size_local : 32544 size in s3 : 32544 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:15 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:09:52 mean.npy already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/synset_words.txt size_local : 334 size in s3 : 334 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:10:06 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt caffemodel_filename : /data/models_weight/learn_rubbia_refus_2500/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10555 max_wait_temp : 1 max_wait : 0 dict_keys(['pool5', 'prob']) time used to do the prepocess of the images : 0.5988078117370605 time used to do the prediction : 0.4677469730377197 save descriptor for thcl : 3237 time to traite the descriptors : 0.6771092414855957 Catched exception ! Connect or reconnect ! storage_type for insertDescriptorsMulti : 3 To insert : 1337185964 To insert : 1337185966 To insert : 1337185968 To insert : 1337185970 To insert : 1337185972 To insert : 1337185974 To insert : 1337185976 To insert : 1337185978 To insert : 1337185980 To insert : 1337185982 To insert : 1337185984 To insert : 1337185986 To insert : 1337185842 To insert : 1337185844 To insert : 1337185846 To insert : 1337185848 To insert : 1337185850 To insert : 1337185852 To insert : 1337185854 To insert : 1337185856 To insert : 1337185858 To insert : 1337185860 To insert : 1337185862 To insert : 1337185864 To insert : 1337186249 To insert : 1337186251 To insert : 1337186253 To insert : 1337186255 To insert : 1337186257 To insert : 1337186259 To insert : 1337186261 To insert : 1337186263 To insert : 1337186265 To insert : 1337186267 To insert : 1337185916 To insert : 1337185918 To insert : 1337185920 To insert : 1337185922 To insert : 1337185924 To insert : 1337185926 To insert : 1337185928 To insert : 1337185930 To insert : 1337185932 To insert : 1337185934 To insert : 1337185936 To insert : 1337185938 To insert : 1337185890 To insert : 1337185892 To insert : 1337185895 To insert : 1337185898 To insert : 1337185900 To insert : 1337185902 To insert : 1337185904 To insert : 1337185906 To insert : 1337185908 To insert : 1337185910 To insert : 1337185912 To insert : 1337185914 To insert : 1337186137 To insert : 1337186159 To insert : 1337186161 To insert : 1337186163 To insert : 1337186165 To insert : 1337186235 To insert : 1337186237 To insert : 1337186239 To insert : 1337186241 To insert : 1337186243 To insert : 1337186245 To insert : 1337186247 To insert : 1337186108 To insert : 1337186111 To insert : 1337186115 To insert : 1337186119 To insert : 1337186121 To insert : 1337186123 To insert : 1337186125 To insert : 1337186127 To insert : 1337186129 To insert : 1337186131 To insert : 1337186133 To insert : 1337186135 To insert : 1337185940 To insert : 1337185942 To insert : 1337185944 To insert : 1337185946 To insert : 1337185948 To insert : 1337185950 To insert : 1337185952 To insert : 1337185954 To insert : 1337185956 To insert : 1337185958 To insert : 1337185960 To insert : 1337185962 To insert : 1337185866 To insert : 1337185868 To insert : 1337185870 To insert : 1337185872 To insert : 1337185874 To insert : 1337185876 To insert : 1337185878 To insert : 1337185880 To insert : 1337185882 To insert : 1337185884 To insert : 1337185886 To insert : 1337185888 To insert : 1337185988 To insert : 1337185990 To insert : 1337185992 To insert : 1337185994 To insert : 1337185996 To insert : 1337186093 To insert : 1337186095 To insert : 1337186097 To insert : 1337186099 To insert : 1337186101 To insert : 1337186103 To insert : 1337186105 time to insert the descriptors : 20.096275806427002 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos SAVE THCL : begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 180 time used for this insertion : 0.020392894744873047 save missing photos in datou_result : time spend for datou_step_exec : 25.94837474822998 time spend to save output : 0.06619477272033691 total time spend for step 6 : 26.014569520950317 step7:ventilate_hashtags_in_portfolio Wed Feb 12 21:22:02 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 20479191 get user id for portfolio 20479191 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20479191 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_fonce','Papier','pehd','Tetrapak','refus','mal_croppe','papier','environnement','flou','PEHD','Carton','autre','metal','carton','Film_plastique','PET_clair','PET_fonce','pet_clair')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20479191 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_fonce','Papier','pehd','Tetrapak','refus','mal_croppe','papier','environnement','flou','PEHD','Carton','autre','metal','carton','Film_plastique','PET_clair','PET_fonce','pet_clair')) AND mptpi.`min_score`=0.6 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20479191 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_fonce','Papier','pehd','Tetrapak','refus','mal_croppe','papier','environnement','flou','PEHD','Carton','autre','metal','carton','Film_plastique','PET_clair','PET_fonce','pet_clair')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20479191 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_fonce','Papier','pehd','Tetrapak','refus','mal_croppe','papier','environnement','flou','PEHD','Carton','autre','metal','carton','Film_plastique','PET_clair','PET_fonce','pet_clair')) AND mptpi.`min_score`=0.6 To do lien utilise dans velours : https://www.fotonower.com/velours/20492040,20492030,20492033,20492027,20492028,20492029,20492031,20492032,20492037,20492035,20492036,20492038,20492041?tags=pet_fonce,papier,pehd,tetrapak,refus,mal_croppe,environnement,flou,carton,autre,metal,film_plastique,pet_clair&datou_id_consolidate=4235&port_consolidate=20479191 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1337117278, 1337117273] Looping around the photos to save general results len do output : 1 /20479191. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2579684') ('4234', '20479191', '1337117278', None, None, None, None, None, '2579684') ('4234', None, None, None, None, None, None, None, '2579684') ('4234', '20479191', '1337117273', None, None, None, None, None, '2579684') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.018872499465942383 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.536583185195923 time spend to save output : 0.019086837768554688 total time spend for step 7 : 2.5556700229644775 step8:final Wed Feb 12 21:22:04 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1337117278: ('0.16684073798992807',), 1337117273: ('0.16684073798992807',)} new output for save of step final : {1337117278: ('0.16684073798992807',), 1337117273: ('0.16684073798992807',)} [1337117278, 1337117273] Looping around the photos to save general results len do output : 2 /1337117278.Didn't retrieve data . /1337117273.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2579684') ('4234', '20479191', '1337117278', None, None, None, None, None, '2579684') ('4234', None, None, None, None, None, None, None, '2579684') ('4234', '20479191', '1337117273', None, None, None, None, None, '2579684') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.013077259063720703 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.11869430541992188 time spend to save output : 0.013293027877807617 total time spend for step 8 : 0.1319873332977295 step9:velours_tree Wed Feb 12 21:22:05 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure list_portfolios : 20492040,20492030,20492033,20492027,20492028,20492029,20492031,20492032,20492037,20492035,20492036,20492038,20492041 photo desc type : 5561 - Retrieving photos to tag... query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20492040 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20492030 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20492033 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20492027 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20492028 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20492029 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20492031 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20492032 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20492037 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20492035 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20492036 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20492038 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20492041 ORDER BY ph.size desc - Loading descriptors... Size : 2048 len(descriptors) : 118 Compute structured hierarchical clustering... ward : AgglomerativeClustering(n_clusters=6) ward.labels_ : [1 1 3 3 2 0 5 5 2 0 1 0 0 0 3 0 1 3 1 3 2 1 2 0 3 2 3 3 0 0 1 2 3 2 2 3 0 0 1 2 1 1 1 3 2 2 3 5 2 5 1 0 1 2 1 1 0 3 0 3 1 1 2 3 3 2 3 3 0 1 2 3 2 2 3 3 1 4 3 1 2 0 0 3 3 5 3 0 4 3 2 0 1 0 3 5 0 1 0 0 0 2 2 4 3 3 0 0 2 3 2 4 2 2 0 0 3 2] Elapsed time: 0.01798701286315918 graph_id used : 78228 - Beta version, working pretty good on 11-5-16 ! https://www.fotonower.com/velours/20492040,20492030,20492033,20492027,20492028,20492029,20492031,20492032,20492037,20492035,20492036,20492038,20492041?tags=pet_fonce,papier,pehd,tetrapak,refus,mal_croppe,environnement,flou,carton,autre,metal,film_plastique,pet_clair&datou_id_consolidate=4235&port_consolidate=20479191&tree_id=78228 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : velours_tree we use saveGeneral [1337117278, 1337117273] Looping around the photos to save general results len do output : 1 /20479191Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2579684') ('4234', '20479191', '1337117278', None, None, None, None, None, '2579684') ('4234', None, None, None, None, None, None, None, '2579684') ('4234', '20479191', '1337117273', None, None, None, None, None, '2579684') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.01573967933654785 save_final save missing photos in datou_result : time spend for datou_step_exec : 8.295090675354004 time spend to save output : 0.015903472900390625 total time spend for step 9 : 8.310994148254395 step10:send_mail_cod Wed Feb 12 21:22:13 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Qualipapia_P20479191_12-02-2025_21_22_13.pdf 20492022 imagette204920221739391733 20492012 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204920121739391733 20492015 imagette204920151739391734 20492009 imagette204920091739391734 20492010 imagette204920101739391734 20492011 imagette204920111739391734 20492014 imagette204920141739391734 20492019 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204920191739391734 20492017 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204920171739391735 20492018 imagette204920181739391736 20492020 imagette204920201739391736 20492023 change filename to text .change filename to text .change filename to text .change filename to text .imagette204920231739391736 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20479191 and hashtag_type = 4230 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20492040,20492030,20492033,20492027,20492028,20492029,20492031,20492032,20492037,20492035,20492036,20492038,20492041?tags=pet_fonce,papier,pehd,tetrapak,refus,mal_croppe,environnement,flou,carton,autre,metal,film_plastique,pet_clair&datou_id_consolidate=4235&port_consolidate=20479191&tree_id=78228 your option no_mail is active, we will not send the real mail to your client args[1337117278] : ((1337117278, -7.813817636105352, 492609224), (1337117278, -0.2794242347148256, 496442774), '0.16684073798992807') apple ((1337117278, -7.813817636105352, 492609224), (1337117278, -0.2794242347148256, 496442774), '0.16684073798992807') We are sending mail with results at cod@fotonower.com args[1337117273] : ((1337117273, -7.83440878842373, 492609224), (1337117273, -0.29597445554122953, 496442774), '0.16684073798992807') apple ((1337117273, -7.83440878842373, 492609224), (1337117273, -0.29597445554122953, 496442774), '0.16684073798992807') We are sending mail with results at cod@fotonower.com refus_total : 0.16684073798992807 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20479191 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20479191_12-02-2025_21_22_13.pdf results_Qualipapia_P20479191_12-02-2025_21_22_13.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20479191_12-02-2025_21_22_13.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4234','20479191','results_Qualipapia_P20479191_12-02-2025_21_22_13.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20479191_12-02-2025_21_22_13.pdf','pdf','','0.13','0.16684073798992807') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1337117278, 1337117273] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2579684') ('4234', '20479191', '1337117278', None, None, None, None, None, '2579684') ('4234', None, None, None, None, None, None, None, '2579684') ('4234', '20479191', '1337117273', None, None, None, None, None, '2579684') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 2 time used for this insertion : 0.01480722427368164 save_final save missing photos in datou_result : time spend for datou_step_exec : 3.7749223709106445 time spend to save output : 0.014954090118408203 total time spend for step 10 : 3.7898764610290527 step11:split_time_score Wed Feb 12 21:22:17 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 3442, 'mtr_user_id': 31, 'name': 'classifieur_2camions_valcor_021122_v1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'deux_camions,camion_droite,camion_gauche,pas_de_camion', 'svm_portfolios_learning': '7659379,7659034,7657685,7657114', 'photo_hashtag_type': 4458, 'photo_desc_type': 5723, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107760533,2107760534,2107760535,2107760536'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('13', 2),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 12022025 20479191 Nombre de photos uploadées : 2 / 23040 (0%) 12022025 20479191 Nombre de photos taguées (types de déchets): 2 / 2 (100%) 12022025 20479191 Nombre de photos taguées (volume) : 0 / 2 (0%) elapsed_time : load_data_split_time_score 1.6689300537109375e-06 elapsed_time : order_list_meta_photo_and_scores 5.0067901611328125e-06 ?? elapsed_time : fill_and_build_computed_from_old_data 0.00019741058349609375 elapsed_time : insert_dashboard_record_day_entry 0.029715299606323242 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.15829996426485274 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20470706_12-02-2025_10_17_40.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20470706 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20470706 AND mptpi.`type`=4230 To do Qualite : 0.2571924266909986 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20474222_12-02-2025_11_47_45.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20474222 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20474222 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476682 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476683 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476684 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476685 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476686 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476321 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476322 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476323 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476324 order by id desc limit 1 Qualite : 0.17840314144925185 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20476325_12-02-2025_12_46_55.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476325 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20476325 AND mptpi.`type`=4230 To do Qualite : 0.23342179416285125 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20476326_12-02-2025_12_36_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476326 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20476326 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476687 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476688 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476689 order by id desc limit 1 Qualite : 0.1924295939903281 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20476690_12-02-2025_13_32_08.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476690 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20476690 AND mptpi.`type`=4230 To do Qualite : 0.2119568776018645 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20476691_12-02-2025_13_23_51.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476691 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20476691 AND mptpi.`type`=4230 To do Qualite : 0.154875790367845 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20476692_12-02-2025_13_17_01.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476692 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20476692 AND mptpi.`type`=4230 To do Qualite : 0.19008527192762648 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20477040_12-02-2025_13_51_47.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20477040 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20477040 AND mptpi.`type`=4230 To do Qualite : 0.19050715192060547 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20477041_12-02-2025_13_39_05.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20477041 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20477041 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20477469 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20477470 order by id desc limit 1 Qualite : 0.22219744748095124 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20477471_12-02-2025_15_33_55.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20477471 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20477471 AND mptpi.`type`=4230 To do Qualite : 0.213374648572307 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20477472_12-02-2025_14_47_07.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20477472 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20477472 AND mptpi.`type`=4230 To do Qualite : 0.2080923629286091 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20477473_12-02-2025_14_17_51.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20477473 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20477473 AND mptpi.`type`=4230 To do Qualite : 0.21079772282942882 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20477474_12-02-2025_14_32_10.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20477474 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20477474 AND mptpi.`type`=4230 To do Qualite : 0.17403366428208897 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20479190_12-02-2025_20_47_39.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479190 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20479190 AND mptpi.`type`=4230 To do Qualite : 0.16684073798992807 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20479191_12-02-2025_21_22_13.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479191 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20479191 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479192 order by id desc limit 1 Qualite : 0.2069816810831725 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20479193_12-02-2025_20_34_25.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479193 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20479193 AND mptpi.`type`=4230 To do Qualite : 0.17777201354499714 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20479194_12-02-2025_19_47_35.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479194 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20479194 AND mptpi.`type`=4230 To do Qualite : 0.19610781617180403 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20479195_12-02-2025_18_33_56.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479195 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20479195 AND mptpi.`type`=4230 To do Qualite : 0.20499992835410263 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20479196_12-02-2025_19_32_57.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479196 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20479196 AND mptpi.`type`=4230 To do Qualite : 0.19000694400268564 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20479197_12-02-2025_16_50_30.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479197 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20479197 AND mptpi.`type`=4230 To do Qualite : 0.23257386904434174 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20479199_12-02-2025_17_34_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479199 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20479199 AND mptpi.`type`=4230 To do Qualite : 0.26488945635035177 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20479200_12-02-2025_17_47_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479200 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20479200 AND mptpi.`type`=4230 To do Qualite : 0.20732146131397367 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20479201_12-02-2025_15_48_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479201 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20479201 AND mptpi.`type`=4230 To do Qualite : 0.22406120432686213 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20479443_12-02-2025_16_22_41.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479443 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20479443 AND mptpi.`type`=4230 To do Qualite : 0.1665337497911145 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20479444_12-02-2025_16_17_51.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479444 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20479444 AND mptpi.`type`=4230 To do Qualite : 0.14940220555870873 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20479445_12-02-2025_16_04_36.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479445 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20479445 AND mptpi.`type`=4230 To do Qualite : 0.2168844323365262 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20485361_12-02-2025_18_34_52.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20485361 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20485361 AND mptpi.`type`=4230 To do Qualite : 0.17109534603709378 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20485362_12-02-2025_19_22_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20485362 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20485362 AND mptpi.`type`=4230 To do Qualite : 0.18625144427744286 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20485363_12-02-2025_18_49_17.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20485363 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20485363 AND mptpi.`type`=4230 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'12022025': {'nb_upload': 2, 'nb_taggue_class': 2, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1337117278, 1337117273] Looping around the photos to save general results len do output : 1 /20479191Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2579684') ('4234', '20479191', '1337117278', None, None, None, None, None, '2579684') ('4234', None, None, None, None, None, None, None, '2579684') ('4234', '20479191', '1337117273', None, None, None, None, None, '2579684') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.0153045654296875 save_final save missing photos in datou_result : time spend for datou_step_exec : 13.264904975891113 time spend to save output : 0.015485048294067383 total time spend for step 11 : 13.28039002418518 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 1 len(input) + len(total_photo_id_missing) : 2 set_done_treatment 31.91user 23.49system 2:04.33elapsed 44%CPU (0avgtext+0avgdata 2504832maxresident)k 518312inputs+10288outputs (270major+1939222minor)pagefaults 0swaps