python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 4234 -l 5 ' -s datou_current_4234 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3432003 load datou : 4234 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load THCL from format json or kwargs add thcl : 3237 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig add pdt : 5561 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4234, datou_cur_ids : ['2654697'] with mtr_portfolio_ids : ['21225802'] and first list_photo_ids : [] new path : /proc/3432003/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, brightness, blur_detection, rle_unique_nms_with_priority, crop_condition, thcl, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , BFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 2 ; length of list_pids : 2 ; length of list_args : 2 time to download the photos : 0.28283190727233887 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Mon Mar 10 16:20:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10151 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-03-10 16:20:31.580517: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-03-10 16:20:31.607307: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-03-10 16:20:31.608929: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f744c000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-03-10 16:20:31.608980: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-03-10 16:20:31.611931: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-03-10 16:20:31.839370: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2b2fa270 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-03-10 16:20:31.839434: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-03-10 16:20:31.840582: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-10 16:20:31.840903: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-10 16:20:31.843136: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-10 16:20:31.845549: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-10 16:20:31.846007: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-10 16:20:31.848619: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-10 16:20:31.849925: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-10 16:20:31.856694: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-10 16:20:31.858353: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-10 16:20:31.858472: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-10 16:20:31.859294: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-10 16:20:31.859316: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-10 16:20:31.859326: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-10 16:20:31.860724: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9399 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-03-10 16:20:32.230462: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-10 16:20:32.230571: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-10 16:20:32.230603: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-10 16:20:32.230633: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-10 16:20:32.230661: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-10 16:20:32.230690: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-10 16:20:32.230718: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-10 16:20:32.230747: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-10 16:20:32.234636: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-10 16:20:32.236128: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-10 16:20:32.236186: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-10 16:20:32.236202: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-10 16:20:32.236216: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-10 16:20:32.236231: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-10 16:20:32.236249: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-10 16:20:32.236267: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-10 16:20:32.236284: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-10 16:20:32.237539: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-10 16:20:32.237572: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-10 16:20:32.237582: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-10 16:20:32.237589: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-10 16:20:32.238944: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9399 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-03-10 16:20:40.184690: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-10 16:20:40.387906: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 2 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -119.46563 max: 149.87734 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 7.319450378417969e-05 nb_pixel_total : 262 time to create 1 rle with old method : 0.0005872249603271484 length of segment : 38 time for calcul the mask position with numpy : 4.458427429199219e-05 nb_pixel_total : 124 time to create 1 rle with old method : 0.00027179718017578125 length of segment : 32 time for calcul the mask position with numpy : 5.1975250244140625e-05 nb_pixel_total : 646 time to create 1 rle with old method : 0.0012788772583007812 length of segment : 51 time for calcul the mask position with numpy : 5.555152893066406e-05 nb_pixel_total : 633 time to create 1 rle with old method : 0.008382081985473633 length of segment : 50 time for calcul the mask position with numpy : 5.269050598144531e-05 nb_pixel_total : 682 time to create 1 rle with old method : 0.0013017654418945312 length of segment : 50 time for calcul the mask position with numpy : 5.555152893066406e-05 nb_pixel_total : 266 time to create 1 rle with old method : 0.00041961669921875 length of segment : 27 time for calcul the mask position with numpy : 5.459785461425781e-05 nb_pixel_total : 118 time to create 1 rle with old method : 0.0003333091735839844 length of segment : 31 time for calcul the mask position with numpy : 9.012222290039062e-05 nb_pixel_total : 2087 time to create 1 rle with old method : 0.0034825801849365234 length of segment : 31 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.29766 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 4.935264587402344e-05 nb_pixel_total : 1054 time to create 1 rle with old method : 0.0013468265533447266 length of segment : 41 time for calcul the mask position with numpy : 4.220008850097656e-05 nb_pixel_total : 904 time to create 1 rle with old method : 0.0012392997741699219 length of segment : 57 time for calcul the mask position with numpy : 3.5762786865234375e-05 nb_pixel_total : 162 time to create 1 rle with old method : 0.0002703666687011719 length of segment : 30 time for calcul the mask position with numpy : 3.457069396972656e-05 nb_pixel_total : 283 time to create 1 rle with old method : 0.0004096031188964844 length of segment : 37 Processing 1 images image shape: (400, 400, 3) min: 21.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -96.25859 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 7.605552673339844e-05 nb_pixel_total : 1979 time to create 1 rle with old method : 0.0027849674224853516 length of segment : 34 time for calcul the mask position with numpy : 4.3392181396484375e-05 nb_pixel_total : 475 time to create 1 rle with old method : 0.0006911754608154297 length of segment : 20 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -121.02422 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 4.3392181396484375e-05 nb_pixel_total : 447 time to create 1 rle with old method : 0.0006325244903564453 length of segment : 23 time for calcul the mask position with numpy : 3.552436828613281e-05 nb_pixel_total : 249 time to create 1 rle with old method : 0.00036334991455078125 length of segment : 23 time for calcul the mask position with numpy : 5.14984130859375e-05 nb_pixel_total : 781 time to create 1 rle with old method : 0.001050710678100586 length of segment : 40 time for calcul the mask position with numpy : 9.202957153320312e-05 nb_pixel_total : 3173 time to create 1 rle with old method : 0.0038602352142333984 length of segment : 117 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 459 time to create 1 rle with old method : 0.0007510185241699219 length of segment : 26 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 451 time to create 1 rle with old method : 0.0007815361022949219 length of segment : 26 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 146.35391 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 4.029273986816406e-05 nb_pixel_total : 391 time to create 1 rle with old method : 0.00058746337890625 length of segment : 15 time for calcul the mask position with numpy : 0.0006196498870849609 nb_pixel_total : 54693 time to create 1 rle with old method : 0.06459546089172363 length of segment : 331 time for calcul the mask position with numpy : 8.153915405273438e-05 nb_pixel_total : 968 time to create 1 rle with old method : 0.0011529922485351562 length of segment : 105 time for calcul the mask position with numpy : 5.9604644775390625e-05 nb_pixel_total : 1318 time to create 1 rle with old method : 0.0015902519226074219 length of segment : 79 time for calcul the mask position with numpy : 4.172325134277344e-05 nb_pixel_total : 355 time to create 1 rle with old method : 0.00048422813415527344 length of segment : 28 time for calcul the mask position with numpy : 3.361701965332031e-05 nb_pixel_total : 67 time to create 1 rle with old method : 0.0001347064971923828 length of segment : 8 time for calcul the mask position with numpy : 4.553794860839844e-05 nb_pixel_total : 961 time to create 1 rle with old method : 0.0012946128845214844 length of segment : 32 time for calcul the mask position with numpy : 4.57763671875e-05 nb_pixel_total : 519 time to create 1 rle with old method : 0.0006856918334960938 length of segment : 35 Processing 1 images image shape: (400, 400, 3) min: 18.00000 max: 196.00000 molded_images shape: (1, 640, 640, 3) min: -95.11016 max: 75.43984 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -113.89531 max: 150.26797 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 0.00020051002502441406 nb_pixel_total : 5789 time to create 1 rle with old method : 0.009961128234863281 length of segment : 334 time for calcul the mask position with numpy : 5.221366882324219e-05 nb_pixel_total : 193 time to create 1 rle with old method : 0.00029015541076660156 length of segment : 18 time for calcul the mask position with numpy : 8.249282836914062e-05 nb_pixel_total : 3168 time to create 1 rle with old method : 0.003947734832763672 length of segment : 65 time for calcul the mask position with numpy : 5.030632019042969e-05 nb_pixel_total : 1230 time to create 1 rle with old method : 0.0016903877258300781 length of segment : 33 time for calcul the mask position with numpy : 3.3855438232421875e-05 nb_pixel_total : 230 time to create 1 rle with old method : 0.00041103363037109375 length of segment : 20 time for calcul the mask position with numpy : 3.409385681152344e-05 nb_pixel_total : 139 time to create 1 rle with old method : 0.00023221969604492188 length of segment : 21 time for calcul the mask position with numpy : 3.0517578125e-05 nb_pixel_total : 125 time to create 1 rle with old method : 0.0002396106719970703 length of segment : 8 time for calcul the mask position with numpy : 6.699562072753906e-05 nb_pixel_total : 2310 time to create 1 rle with old method : 0.0027735233306884766 length of segment : 45 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -113.61016 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 6.341934204101562e-05 nb_pixel_total : 139 time to create 1 rle with old method : 0.00039458274841308594 length of segment : 13 time for calcul the mask position with numpy : 8.511543273925781e-05 nb_pixel_total : 222 time to create 1 rle with old method : 0.0004825592041015625 length of segment : 25 time for calcul the mask position with numpy : 0.0004825592041015625 nb_pixel_total : 16224 time to create 1 rle with old method : 0.02900838851928711 length of segment : 220 time for calcul the mask position with numpy : 0.00011587142944335938 nb_pixel_total : 645 time to create 1 rle with old method : 0.0012781620025634766 length of segment : 32 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -118.91484 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 10 time for calcul the mask position with numpy : 8.916854858398438e-05 nb_pixel_total : 1785 time to create 1 rle with old method : 0.0024755001068115234 length of segment : 73 time for calcul the mask position with numpy : 0.00014328956604003906 nb_pixel_total : 3018 time to create 1 rle with old method : 0.0038199424743652344 length of segment : 160 time for calcul the mask position with numpy : 4.6253204345703125e-05 nb_pixel_total : 103 time to create 1 rle with old method : 0.0001888275146484375 length of segment : 10 time for calcul the mask position with numpy : 9.846687316894531e-05 nb_pixel_total : 1651 time to create 1 rle with old method : 0.0021300315856933594 length of segment : 105 time for calcul the mask position with numpy : 4.482269287109375e-05 nb_pixel_total : 179 time to create 1 rle with old method : 0.0003292560577392578 length of segment : 12 time for calcul the mask position with numpy : 6.413459777832031e-05 nb_pixel_total : 1083 time to create 1 rle with old method : 0.002067089080810547 length of segment : 24 time for calcul the mask position with numpy : 0.0003216266632080078 nb_pixel_total : 6322 time to create 1 rle with old method : 0.007970094680786133 length of segment : 198 time for calcul the mask position with numpy : 8.940696716308594e-05 nb_pixel_total : 290 time to create 1 rle with old method : 0.0006017684936523438 length of segment : 19 time for calcul the mask position with numpy : 0.00014066696166992188 nb_pixel_total : 2942 time to create 1 rle with old method : 0.004991769790649414 length of segment : 106 time for calcul the mask position with numpy : 5.9604644775390625e-05 nb_pixel_total : 60 time to create 1 rle with old method : 0.000179290771484375 length of segment : 6 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 243.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 121.93203 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 8.7738037109375e-05 nb_pixel_total : 2107 time to create 1 rle with old method : 0.0028204917907714844 length of segment : 189 time for calcul the mask position with numpy : 9.393692016601562e-05 nb_pixel_total : 1706 time to create 1 rle with old method : 0.0021169185638427734 length of segment : 78 time for calcul the mask position with numpy : 0.00011897087097167969 nb_pixel_total : 2591 time to create 1 rle with old method : 0.003378629684448242 length of segment : 170 time for calcul the mask position with numpy : 0.0002567768096923828 nb_pixel_total : 3438 time to create 1 rle with old method : 0.004406929016113281 length of segment : 193 Processing 1 images image shape: (280, 400, 3) min: 12.00000 max: 178.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 61.74062 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0012841224670410156 nb_pixel_total : 104756 time to create 1 rle with old method : 0.16053247451782227 length of segment : 282 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 144.01797 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 9.512901306152344e-05 nb_pixel_total : 808 time to create 1 rle with old method : 0.0016415119171142578 length of segment : 35 time for calcul the mask position with numpy : 5.817413330078125e-05 nb_pixel_total : 353 time to create 1 rle with old method : 0.0007455348968505859 length of segment : 23 time for calcul the mask position with numpy : 4.410743713378906e-05 nb_pixel_total : 121 time to create 1 rle with old method : 0.00032901763916015625 length of segment : 15 time for calcul the mask position with numpy : 5.650520324707031e-05 nb_pixel_total : 145 time to create 1 rle with old method : 0.0003039836883544922 length of segment : 14 time for calcul the mask position with numpy : 3.600120544433594e-05 nb_pixel_total : 318 time to create 1 rle with old method : 0.0005841255187988281 length of segment : 17 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 148.33828 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 16 time for calcul the mask position with numpy : 8.034706115722656e-05 nb_pixel_total : 281 time to create 1 rle with old method : 0.0004134178161621094 length of segment : 40 time for calcul the mask position with numpy : 5.078315734863281e-05 nb_pixel_total : 296 time to create 1 rle with old method : 0.00042366981506347656 length of segment : 26 time for calcul the mask position with numpy : 4.887580871582031e-05 nb_pixel_total : 99 time to create 1 rle with old method : 0.00019073486328125 length of segment : 10 time for calcul the mask position with numpy : 8.487701416015625e-05 nb_pixel_total : 87 time to create 1 rle with old method : 0.00015425682067871094 length of segment : 14 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 262 time to create 1 rle with old method : 0.0004074573516845703 length of segment : 30 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 363 time to create 1 rle with old method : 0.0005536079406738281 length of segment : 18 time for calcul the mask position with numpy : 3.4332275390625e-05 nb_pixel_total : 86 time to create 1 rle with old method : 0.0001556873321533203 length of segment : 10 time for calcul the mask position with numpy : 3.838539123535156e-05 nb_pixel_total : 148 time to create 1 rle with old method : 0.00027370452880859375 length of segment : 14 time for calcul the mask position with numpy : 4.482269287109375e-05 nb_pixel_total : 318 time to create 1 rle with old method : 0.0006387233734130859 length of segment : 25 time for calcul the mask position with numpy : 8.535385131835938e-05 nb_pixel_total : 495 time to create 1 rle with old method : 0.0008082389831542969 length of segment : 21 time for calcul the mask position with numpy : 5.53131103515625e-05 nb_pixel_total : 58 time to create 1 rle with old method : 0.00017404556274414062 length of segment : 6 time for calcul the mask position with numpy : 6.532669067382812e-05 nb_pixel_total : 409 time to create 1 rle with old method : 0.0009338855743408203 length of segment : 16 time for calcul the mask position with numpy : 5.1975250244140625e-05 nb_pixel_total : 254 time to create 1 rle with old method : 0.0005249977111816406 length of segment : 17 time for calcul the mask position with numpy : 7.05718994140625e-05 nb_pixel_total : 543 time to create 1 rle with old method : 0.0011129379272460938 length of segment : 48 time for calcul the mask position with numpy : 5.841255187988281e-05 nb_pixel_total : 168 time to create 1 rle with old method : 0.0004992485046386719 length of segment : 17 time for calcul the mask position with numpy : 7.128715515136719e-05 nb_pixel_total : 895 time to create 1 rle with old method : 0.001748800277709961 length of segment : 57 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.24062 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 22 time for calcul the mask position with numpy : 7.319450378417969e-05 nb_pixel_total : 56 time to create 1 rle with old method : 0.00012969970703125 length of segment : 8 time for calcul the mask position with numpy : 4.410743713378906e-05 nb_pixel_total : 383 time to create 1 rle with old method : 0.0005195140838623047 length of segment : 35 time for calcul the mask position with numpy : 5.91278076171875e-05 nb_pixel_total : 1125 time to create 1 rle with old method : 0.0016334056854248047 length of segment : 29 time for calcul the mask position with numpy : 4.839897155761719e-05 nb_pixel_total : 285 time to create 1 rle with old method : 0.00040030479431152344 length of segment : 24 time for calcul the mask position with numpy : 4.1484832763671875e-05 nb_pixel_total : 222 time to create 1 rle with old method : 0.0003147125244140625 length of segment : 23 time for calcul the mask position with numpy : 4.9114227294921875e-05 nb_pixel_total : 421 time to create 1 rle with old method : 0.000644683837890625 length of segment : 23 time for calcul the mask position with numpy : 6.961822509765625e-05 nb_pixel_total : 61 time to create 1 rle with old method : 0.00015664100646972656 length of segment : 9 time for calcul the mask position with numpy : 3.5762786865234375e-05 nb_pixel_total : 66 time to create 1 rle with old method : 0.00012564659118652344 length of segment : 12 time for calcul the mask position with numpy : 5.245208740234375e-05 nb_pixel_total : 536 time to create 1 rle with old method : 0.0007679462432861328 length of segment : 35 time for calcul the mask position with numpy : 0.00010013580322265625 nb_pixel_total : 2354 time to create 1 rle with old method : 0.002931833267211914 length of segment : 77 time for calcul the mask position with numpy : 5.650520324707031e-05 nb_pixel_total : 708 time to create 1 rle with old method : 0.0010395050048828125 length of segment : 23 time for calcul the mask position with numpy : 4.649162292480469e-05 nb_pixel_total : 73 time to create 1 rle with old method : 0.0001595020294189453 length of segment : 6 time for calcul the mask position with numpy : 5.221366882324219e-05 nb_pixel_total : 413 time to create 1 rle with old method : 0.0005817413330078125 length of segment : 29 time for calcul the mask position with numpy : 3.528594970703125e-05 nb_pixel_total : 124 time to create 1 rle with old method : 0.00019979476928710938 length of segment : 13 time for calcul the mask position with numpy : 4.8160552978515625e-05 nb_pixel_total : 664 time to create 1 rle with old method : 0.0008704662322998047 length of segment : 37 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 468 time to create 1 rle with old method : 0.0006241798400878906 length of segment : 31 time for calcul the mask position with numpy : 3.6716461181640625e-05 nb_pixel_total : 81 time to create 1 rle with old method : 0.00013971328735351562 length of segment : 11 time for calcul the mask position with numpy : 4.673004150390625e-05 nb_pixel_total : 213 time to create 1 rle with old method : 0.0003566741943359375 length of segment : 35 time for calcul the mask position with numpy : 7.581710815429688e-05 nb_pixel_total : 766 time to create 1 rle with old method : 0.0010616779327392578 length of segment : 47 time for calcul the mask position with numpy : 6.794929504394531e-05 nb_pixel_total : 920 time to create 1 rle with old method : 0.0012524127960205078 length of segment : 58 time for calcul the mask position with numpy : 0.00010776519775390625 nb_pixel_total : 2085 time to create 1 rle with old method : 0.0032012462615966797 length of segment : 39 time for calcul the mask position with numpy : 0.0001392364501953125 nb_pixel_total : 2319 time to create 1 rle with old method : 0.0030248165130615234 length of segment : 78 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 138.78750 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 7.343292236328125e-05 nb_pixel_total : 187 time to create 1 rle with old method : 0.0003094673156738281 length of segment : 15 time for calcul the mask position with numpy : 4.887580871582031e-05 nb_pixel_total : 640 time to create 1 rle with old method : 0.0008537769317626953 length of segment : 32 time for calcul the mask position with numpy : 5.650520324707031e-05 nb_pixel_total : 676 time to create 1 rle with old method : 0.0008971691131591797 length of segment : 34 time for calcul the mask position with numpy : 0.00011920928955078125 nb_pixel_total : 273 time to create 1 rle with old method : 0.0004582405090332031 length of segment : 57 time for calcul the mask position with numpy : 7.486343383789062e-05 nb_pixel_total : 432 time to create 1 rle with old method : 0.0007433891296386719 length of segment : 24 time for calcul the mask position with numpy : 6.508827209472656e-05 nb_pixel_total : 769 time to create 1 rle with old method : 0.0010843276977539062 length of segment : 38 time for calcul the mask position with numpy : 5.698204040527344e-05 nb_pixel_total : 208 time to create 1 rle with old method : 0.00040030479431152344 length of segment : 29 time for calcul the mask position with numpy : 0.00010967254638671875 nb_pixel_total : 80 time to create 1 rle with old method : 0.00018739700317382812 length of segment : 27 time for calcul the mask position with numpy : 3.814697265625e-05 nb_pixel_total : 117 time to create 1 rle with old method : 0.0002067089080810547 length of segment : 9 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -119.71172 max: 149.89297 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 8.225440979003906e-05 nb_pixel_total : 630 time to create 1 rle with old method : 0.0009961128234863281 length of segment : 47 time for calcul the mask position with numpy : 7.033348083496094e-05 nb_pixel_total : 701 time to create 1 rle with old method : 0.0010440349578857422 length of segment : 52 time for calcul the mask position with numpy : 7.510185241699219e-05 nb_pixel_total : 648 time to create 1 rle with old method : 0.0010902881622314453 length of segment : 47 time for calcul the mask position with numpy : 6.198883056640625e-05 nb_pixel_total : 133 time to create 1 rle with old method : 0.0002429485321044922 length of segment : 33 time for calcul the mask position with numpy : 4.482269287109375e-05 nb_pixel_total : 276 time to create 1 rle with old method : 0.0004153251647949219 length of segment : 31 time for calcul the mask position with numpy : 4.3392181396484375e-05 nb_pixel_total : 285 time to create 1 rle with old method : 0.00040721893310546875 length of segment : 28 time for calcul the mask position with numpy : 4.7206878662109375e-05 nb_pixel_total : 138 time to create 1 rle with old method : 0.00022220611572265625 length of segment : 33 time for calcul the mask position with numpy : 8.177757263183594e-05 nb_pixel_total : 2209 time to create 1 rle with old method : 0.0029401779174804688 length of segment : 32 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -118.76250 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 7.367134094238281e-05 nb_pixel_total : 1080 time to create 1 rle with old method : 0.0024657249450683594 length of segment : 33 time for calcul the mask position with numpy : 0.00011110305786132812 nb_pixel_total : 3419 time to create 1 rle with old method : 0.0062520503997802734 length of segment : 53 time for calcul the mask position with numpy : 0.00011086463928222656 nb_pixel_total : 967 time to create 1 rle with old method : 0.003917217254638672 length of segment : 37 time for calcul the mask position with numpy : 0.00013208389282226562 nb_pixel_total : 2996 time to create 1 rle with old method : 0.004235744476318359 length of segment : 50 Processing 1 images image shape: (400, 400, 3) min: 26.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -85.23516 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 0.00017070770263671875 nb_pixel_total : 5263 time to create 1 rle with old method : 0.006415843963623047 length of segment : 149 time for calcul the mask position with numpy : 0.00016427040100097656 nb_pixel_total : 1633 time to create 1 rle with old method : 0.003895282745361328 length of segment : 24 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -121.36016 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 7.772445678710938e-05 nb_pixel_total : 662 time to create 1 rle with old method : 0.0012087821960449219 length of segment : 48 time for calcul the mask position with numpy : 8.821487426757812e-05 nb_pixel_total : 559 time to create 1 rle with old method : 0.0010788440704345703 length of segment : 26 time for calcul the mask position with numpy : 0.00011992454528808594 nb_pixel_total : 870 time to create 1 rle with old method : 0.0016188621520996094 length of segment : 36 time for calcul the mask position with numpy : 6.079673767089844e-05 nb_pixel_total : 239 time to create 1 rle with old method : 0.0004684925079345703 length of segment : 24 time for calcul the mask position with numpy : 0.00011014938354492188 nb_pixel_total : 379 time to create 1 rle with old method : 0.0011098384857177734 length of segment : 42 time for calcul the mask position with numpy : 0.0001373291015625 nb_pixel_total : 2568 time to create 1 rle with old method : 0.0041501522064208984 length of segment : 121 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 140.98672 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 7.82012939453125e-05 nb_pixel_total : 445 time to create 1 rle with old method : 0.0009360313415527344 length of segment : 14 time for calcul the mask position with numpy : 8.654594421386719e-05 nb_pixel_total : 1453 time to create 1 rle with old method : 0.0024161338806152344 length of segment : 104 time for calcul the mask position with numpy : 0.0009229183197021484 nb_pixel_total : 61579 time to create 1 rle with old method : 0.08724808692932129 length of segment : 288 time for calcul the mask position with numpy : 0.0005726814270019531 nb_pixel_total : 54849 time to create 1 rle with old method : 0.0660088062286377 length of segment : 244 Processing 1 images image shape: (400, 400, 3) min: 17.00000 max: 192.00000 molded_images shape: (1, 640, 640, 3) min: -95.41484 max: 71.81094 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -112.50469 max: 150.90469 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 4.7206878662109375e-05 nb_pixel_total : 213 time to create 1 rle with old method : 0.0003237724304199219 length of segment : 19 time for calcul the mask position with numpy : 4.57763671875e-05 nb_pixel_total : 120 time to create 1 rle with old method : 0.00025725364685058594 length of segment : 7 time for calcul the mask position with numpy : 4.1961669921875e-05 nb_pixel_total : 179 time to create 1 rle with old method : 0.0002770423889160156 length of segment : 26 time for calcul the mask position with numpy : 9.608268737792969e-05 nb_pixel_total : 3116 time to create 1 rle with old method : 0.004731893539428711 length of segment : 63 time for calcul the mask position with numpy : 4.1961669921875e-05 nb_pixel_total : 212 time to create 1 rle with old method : 0.00039577484130859375 length of segment : 19 time for calcul the mask position with numpy : 4.935264587402344e-05 nb_pixel_total : 661 time to create 1 rle with old method : 0.0009369850158691406 length of segment : 24 time for calcul the mask position with numpy : 7.271766662597656e-05 nb_pixel_total : 548 time to create 1 rle with old method : 0.0011854171752929688 length of segment : 23 time for calcul the mask position with numpy : 9.131431579589844e-05 nb_pixel_total : 2226 time to create 1 rle with old method : 0.0033311843872070312 length of segment : 42 Processing 1 images image shape: (400, 400, 3) min: 2.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -108.91875 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 7.796287536621094e-05 nb_pixel_total : 123 time to create 1 rle with old method : 0.00039005279541015625 length of segment : 11 time for calcul the mask position with numpy : 0.00033092498779296875 nb_pixel_total : 8973 time to create 1 rle with old method : 0.015650033950805664 length of segment : 201 time for calcul the mask position with numpy : 8.988380432128906e-05 nb_pixel_total : 310 time to create 1 rle with old method : 0.0007226467132568359 length of segment : 29 time for calcul the mask position with numpy : 7.510185241699219e-05 nb_pixel_total : 671 time to create 1 rle with old method : 0.0014030933380126953 length of segment : 31 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -118.43828 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 14 time for calcul the mask position with numpy : 0.00017070770263671875 nb_pixel_total : 5813 time to create 1 rle with old method : 0.0068171024322509766 length of segment : 194 time for calcul the mask position with numpy : 3.838539123535156e-05 nb_pixel_total : 92 time to create 1 rle with old method : 0.00017023086547851562 length of segment : 8 time for calcul the mask position with numpy : 3.0994415283203125e-05 nb_pixel_total : 145 time to create 1 rle with old method : 0.000263214111328125 length of segment : 11 time for calcul the mask position with numpy : 3.170967102050781e-05 nb_pixel_total : 100 time to create 1 rle with old method : 0.00016546249389648438 length of segment : 18 time for calcul the mask position with numpy : 8.249282836914062e-05 nb_pixel_total : 1635 time to create 1 rle with old method : 0.0021157264709472656 length of segment : 116 time for calcul the mask position with numpy : 0.0002720355987548828 nb_pixel_total : 5784 time to create 1 rle with old method : 0.007146358489990234 length of segment : 239 time for calcul the mask position with numpy : 7.963180541992188e-05 nb_pixel_total : 1696 time to create 1 rle with old method : 0.0021970272064208984 length of segment : 58 time for calcul the mask position with numpy : 0.00014448165893554688 nb_pixel_total : 2442 time to create 1 rle with old method : 0.0031502246856689453 length of segment : 108 time for calcul the mask position with numpy : 3.933906555175781e-05 nb_pixel_total : 65 time to create 1 rle with old method : 0.00014138221740722656 length of segment : 8 time for calcul the mask position with numpy : 3.719329833984375e-05 nb_pixel_total : 322 time to create 1 rle with old method : 0.00048089027404785156 length of segment : 20 time for calcul the mask position with numpy : 5.173683166503906e-05 nb_pixel_total : 1090 time to create 1 rle with old method : 0.0014584064483642578 length of segment : 39 time for calcul the mask position with numpy : 0.00019621849060058594 nb_pixel_total : 1310 time to create 1 rle with old method : 0.0018811225891113281 length of segment : 31 time for calcul the mask position with numpy : 5.221366882324219e-05 nb_pixel_total : 35 time to create 1 rle with old method : 9.1552734375e-05 length of segment : 8 time for calcul the mask position with numpy : 4.887580871582031e-05 nb_pixel_total : 211 time to create 1 rle with old method : 0.0003478527069091797 length of segment : 37 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 243.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 124.36953 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 0.00020194053649902344 nb_pixel_total : 1638 time to create 1 rle with old method : 0.004042863845825195 length of segment : 153 time for calcul the mask position with numpy : 0.0002224445343017578 nb_pixel_total : 1757 time to create 1 rle with old method : 0.0039141178131103516 length of segment : 82 time for calcul the mask position with numpy : 0.00011610984802246094 nb_pixel_total : 1969 time to create 1 rle with old method : 0.0026247501373291016 length of segment : 168 time for calcul the mask position with numpy : 6.413459777832031e-05 nb_pixel_total : 545 time to create 1 rle with old method : 0.0009336471557617188 length of segment : 35 time for calcul the mask position with numpy : 0.0002522468566894531 nb_pixel_total : 3615 time to create 1 rle with old method : 0.005239009857177734 length of segment : 189 Processing 1 images image shape: (280, 400, 3) min: 11.00000 max: 183.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 65.00625 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0009818077087402344 nb_pixel_total : 105079 time to create 1 rle with old method : 0.1432647705078125 length of segment : 279 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 138.38125 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 6.29425048828125e-05 nb_pixel_total : 886 time to create 1 rle with old method : 0.0012614727020263672 length of segment : 39 time for calcul the mask position with numpy : 6.937980651855469e-05 nb_pixel_total : 289 time to create 1 rle with old method : 0.0006425380706787109 length of segment : 19 time for calcul the mask position with numpy : 5.054473876953125e-05 nb_pixel_total : 343 time to create 1 rle with old method : 0.0005173683166503906 length of segment : 24 time for calcul the mask position with numpy : 6.127357482910156e-05 nb_pixel_total : 684 time to create 1 rle with old method : 0.0009162425994873047 length of segment : 46 time for calcul the mask position with numpy : 6.222724914550781e-05 nb_pixel_total : 617 time to create 1 rle with old method : 0.0008404254913330078 length of segment : 45 time for calcul the mask position with numpy : 6.413459777832031e-05 nb_pixel_total : 176 time to create 1 rle with old method : 0.0004382133483886719 length of segment : 26 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 146.94766 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 13 time for calcul the mask position with numpy : 0.00010061264038085938 nb_pixel_total : 291 time to create 1 rle with old method : 0.0006453990936279297 length of segment : 25 time for calcul the mask position with numpy : 5.1975250244140625e-05 nb_pixel_total : 426 time to create 1 rle with old method : 0.0006124973297119141 length of segment : 43 time for calcul the mask position with numpy : 4.9114227294921875e-05 nb_pixel_total : 342 time to create 1 rle with old method : 0.0005016326904296875 length of segment : 36 time for calcul the mask position with numpy : 5.698204040527344e-05 nb_pixel_total : 255 time to create 1 rle with old method : 0.001430511474609375 length of segment : 19 time for calcul the mask position with numpy : 8.940696716308594e-05 nb_pixel_total : 251 time to create 1 rle with old method : 0.0006031990051269531 length of segment : 13 time for calcul the mask position with numpy : 8.559226989746094e-05 nb_pixel_total : 78 time to create 1 rle with old method : 0.00024175643920898438 length of segment : 12 time for calcul the mask position with numpy : 0.0007224082946777344 nb_pixel_total : 83 time to create 1 rle with old method : 0.002471923828125 length of segment : 10 time for calcul the mask position with numpy : 8.153915405273438e-05 nb_pixel_total : 412 time to create 1 rle with old method : 0.0008928775787353516 length of segment : 24 time for calcul the mask position with numpy : 4.935264587402344e-05 nb_pixel_total : 276 time to create 1 rle with old method : 0.0004229545593261719 length of segment : 25 time for calcul the mask position with numpy : 8.296966552734375e-05 nb_pixel_total : 431 time to create 1 rle with old method : 0.0010104179382324219 length of segment : 23 time for calcul the mask position with numpy : 5.245208740234375e-05 nb_pixel_total : 284 time to create 1 rle with old method : 0.0004546642303466797 length of segment : 26 time for calcul the mask position with numpy : 7.2479248046875e-05 nb_pixel_total : 149 time to create 1 rle with old method : 0.0004215240478515625 length of segment : 13 time for calcul the mask position with numpy : 5.14984130859375e-05 nb_pixel_total : 308 time to create 1 rle with old method : 0.0005240440368652344 length of segment : 25 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.07656 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 19 time for calcul the mask position with numpy : 8.058547973632812e-05 nb_pixel_total : 47 time to create 1 rle with old method : 0.0001647472381591797 length of segment : 7 time for calcul the mask position with numpy : 6.389617919921875e-05 nb_pixel_total : 306 time to create 1 rle with old method : 0.0006630420684814453 length of segment : 22 time for calcul the mask position with numpy : 9.465217590332031e-05 nb_pixel_total : 943 time to create 1 rle with old method : 0.002045154571533203 length of segment : 32 time for calcul the mask position with numpy : 6.866455078125e-05 nb_pixel_total : 240 time to create 1 rle with old method : 0.0005540847778320312 length of segment : 25 time for calcul the mask position with numpy : 5.817413330078125e-05 nb_pixel_total : 34 time to create 1 rle with old method : 0.00015497207641601562 length of segment : 19 time for calcul the mask position with numpy : 7.462501525878906e-05 nb_pixel_total : 526 time to create 1 rle with old method : 0.0010995864868164062 length of segment : 34 time for calcul the mask position with numpy : 7.2479248046875e-05 nb_pixel_total : 723 time to create 1 rle with old method : 0.0016226768493652344 length of segment : 22 time for calcul the mask position with numpy : 5.8650970458984375e-05 nb_pixel_total : 57 time to create 1 rle with old method : 0.0001671314239501953 length of segment : 12 time for calcul the mask position with numpy : 5.984306335449219e-05 nb_pixel_total : 56 time to create 1 rle with old method : 0.00019741058349609375 length of segment : 12 time for calcul the mask position with numpy : 8.463859558105469e-05 nb_pixel_total : 1025 time to create 1 rle with old method : 0.002062559127807617 length of segment : 57 time for calcul the mask position with numpy : 0.00012063980102539062 nb_pixel_total : 2454 time to create 1 rle with old method : 0.004763126373291016 length of segment : 76 time for calcul the mask position with numpy : 7.843971252441406e-05 nb_pixel_total : 346 time to create 1 rle with old method : 0.000728607177734375 length of segment : 32 time for calcul the mask position with numpy : 5.555152893066406e-05 nb_pixel_total : 89 time to create 1 rle with old method : 0.00026154518127441406 length of segment : 7 time for calcul the mask position with numpy : 0.00012111663818359375 nb_pixel_total : 1454 time to create 1 rle with old method : 0.0034034252166748047 length of segment : 30 time for calcul the mask position with numpy : 0.00012087821960449219 nb_pixel_total : 1384 time to create 1 rle with old method : 0.0030465126037597656 length of segment : 35 time for calcul the mask position with numpy : 7.62939453125e-05 nb_pixel_total : 454 time to create 1 rle with old method : 0.0011763572692871094 length of segment : 40 time for calcul the mask position with numpy : 8.893013000488281e-05 nb_pixel_total : 819 time to create 1 rle with old method : 0.0016589164733886719 length of segment : 58 time for calcul the mask position with numpy : 8.535385131835938e-05 nb_pixel_total : 747 time to create 1 rle with old method : 0.0016715526580810547 length of segment : 45 time for calcul the mask position with numpy : 0.00010204315185546875 nb_pixel_total : 2402 time to create 1 rle with old method : 0.0038170814514160156 length of segment : 75 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 139.97500 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 12 time for calcul the mask position with numpy : 7.891654968261719e-05 nb_pixel_total : 185 time to create 1 rle with old method : 0.00041937828063964844 length of segment : 15 time for calcul the mask position with numpy : 7.772445678710938e-05 nb_pixel_total : 351 time to create 1 rle with old method : 0.0007090568542480469 length of segment : 32 time for calcul the mask position with numpy : 7.510185241699219e-05 nb_pixel_total : 672 time to create 1 rle with old method : 0.0012006759643554688 length of segment : 33 time for calcul the mask position with numpy : 5.698204040527344e-05 nb_pixel_total : 406 time to create 1 rle with old method : 0.0007760524749755859 length of segment : 22 time for calcul the mask position with numpy : 0.00010704994201660156 nb_pixel_total : 273 time to create 1 rle with old method : 0.0005998611450195312 length of segment : 57 time for calcul the mask position with numpy : 9.465217590332031e-05 nb_pixel_total : 479 time to create 1 rle with old method : 0.0008938312530517578 length of segment : 27 time for calcul the mask position with numpy : 5.3882598876953125e-05 nb_pixel_total : 116 time to create 1 rle with old method : 0.0002505779266357422 length of segment : 14 time for calcul the mask position with numpy : 5.817413330078125e-05 nb_pixel_total : 635 time to create 1 rle with old method : 0.001104593276977539 length of segment : 30 time for calcul the mask position with numpy : 7.605552673339844e-05 nb_pixel_total : 706 time to create 1 rle with old method : 0.0011980533599853516 length of segment : 35 time for calcul the mask position with numpy : 6.175041198730469e-05 nb_pixel_total : 118 time to create 1 rle with old method : 0.000278472900390625 length of segment : 10 time for calcul the mask position with numpy : 6.866455078125e-05 nb_pixel_total : 74 time to create 1 rle with old method : 0.00019669532775878906 length of segment : 13 time for calcul the mask position with numpy : 5.745887756347656e-05 nb_pixel_total : 107 time to create 1 rle with old method : 0.00024962425231933594 length of segment : 14 Detection mask done ! Trying to reset tf kernel 3432665 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 3126 tf kernel not reseted sub process len(results) : 213 len(list_Values) 213 None max_time_sub_proc : 3600 parent process len(results) : 0 len(list_Values) 213 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 9352 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Catched exception ! Connect or reconnect ! Number saved : None batch 1 Loaded 241 chid ids of type : 4228 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 2340 save missing photos in datou_result : time spend for datou_step_exec : 24.57758617401123 time spend to save output : 0.8252215385437012 total time spend for step 1 : 25.40280771255493 step2:brightness Mon Mar 10 16:20:53 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness toutes les photos sont déjà traitées, on saute les calculs Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 2 time used for this insertion : 0.020787715911865234 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 2 time used for this insertion : 0.021876811981201172 save missing photos in datou_result : time spend for datou_step_exec : 0.0663912296295166 time spend to save output : 0.055084943771362305 total time spend for step 2 : 0.1214761734008789 step3:blur_detection Mon Mar 10 16:20:54 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection toutes les photos sont déjà traitées, on saute les calculs Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 2 time used for this insertion : 0.024264097213745117 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 2 time used for this insertion : 0.024397850036621094 save missing photos in datou_result : time spend for datou_step_exec : 0.058869123458862305 time spend to save output : 0.05892372131347656 total time spend for step 3 : 0.11779284477233887 step4:rle_unique_nms_with_priority Mon Mar 10 16:20:54 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 241 chid ids of type : 4228 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 117 nb_hashtags : 5 time to prepare the origin masks : 1.2491812705993652 time for calcul the mask position with numpy : 0.12585663795471191 nb_pixel_total : 1804697 time to create 1 rle with new method : 0.10198497772216797 time for calcul the mask position with numpy : 0.008574962615966797 nb_pixel_total : 14940 time to create 1 rle with old method : 0.016847610473632812 time for calcul the mask position with numpy : 0.014275789260864258 nb_pixel_total : 54693 time to create 1 rle with old method : 0.06528210639953613 time for calcul the mask position with numpy : 0.008728742599487305 nb_pixel_total : 904 time to create 1 rle with old method : 0.001039266586303711 time for calcul the mask position with numpy : 0.008713722229003906 nb_pixel_total : 123 time to create 1 rle with old method : 0.00016450881958007812 time for calcul the mask position with numpy : 0.00882577896118164 nb_pixel_total : 107 time to create 1 rle with old method : 0.00016188621520996094 time for calcul the mask position with numpy : 0.008722066879272461 nb_pixel_total : 76 time to create 1 rle with old method : 0.00011992454528808594 time for calcul the mask position with numpy : 0.008687973022460938 nb_pixel_total : 646 time to create 1 rle with old method : 0.0007634162902832031 time for calcul the mask position with numpy : 0.008642435073852539 nb_pixel_total : 47 time to create 1 rle with old method : 6.961822509765625e-05 time for calcul the mask position with numpy : 0.008605480194091797 nb_pixel_total : 57 time to create 1 rle with old method : 8.392333984375e-05 time for calcul the mask position with numpy : 0.008781194686889648 nb_pixel_total : 2087 time to create 1 rle with old method : 0.002429485321044922 time for calcul the mask position with numpy : 0.00893402099609375 nb_pixel_total : 162 time to create 1 rle with old method : 0.00020241737365722656 time for calcul the mask position with numpy : 0.008646249771118164 nb_pixel_total : 124 time to create 1 rle with old method : 0.0001533031463623047 time for calcul the mask position with numpy : 0.008509159088134766 nb_pixel_total : 262 time to create 1 rle with old method : 0.0003101825714111328 time for calcul the mask position with numpy : 0.008699178695678711 nb_pixel_total : 781 time to create 1 rle with old method : 0.0009443759918212891 time for calcul the mask position with numpy : 0.00870203971862793 nb_pixel_total : 26 time to create 1 rle with old method : 5.030632019042969e-05 time for calcul the mask position with numpy : 0.008789300918579102 nb_pixel_total : 1572 time to create 1 rle with old method : 0.0018274784088134766 time for calcul the mask position with numpy : 0.008716583251953125 nb_pixel_total : 249 time to create 1 rle with old method : 0.0003046989440917969 time for calcul the mask position with numpy : 0.008880376815795898 nb_pixel_total : 352 time to create 1 rle with old method : 0.00039839744567871094 time for calcul the mask position with numpy : 0.009089231491088867 nb_pixel_total : 1054 time to create 1 rle with old method : 0.001218557357788086 time for calcul the mask position with numpy : 0.008725643157958984 nb_pixel_total : 6 time to create 1 rle with old method : 2.574920654296875e-05 time for calcul the mask position with numpy : 0.008834362030029297 nb_pixel_total : 6 time to create 1 rle with old method : 2.5987625122070312e-05 time for calcul the mask position with numpy : 0.008982658386230469 nb_pixel_total : 359 time to create 1 rle with old method : 0.0004508495330810547 time for calcul the mask position with numpy : 0.008443117141723633 nb_pixel_total : 475 time to create 1 rle with old method : 0.0005419254302978516 time for calcul the mask position with numpy : 0.008511543273925781 nb_pixel_total : 447 time to create 1 rle with old method : 0.0005118846893310547 time for calcul the mask position with numpy : 0.008525371551513672 nb_pixel_total : 19 time to create 1 rle with old method : 7.653236389160156e-05 time for calcul the mask position with numpy : 0.00909566879272461 nb_pixel_total : 459 time to create 1 rle with old method : 0.0006289482116699219 time for calcul the mask position with numpy : 0.009100675582885742 nb_pixel_total : 968 time to create 1 rle with old method : 0.0011420249938964844 time for calcul the mask position with numpy : 0.010098934173583984 nb_pixel_total : 1982 time to create 1 rle with old method : 0.0036640167236328125 time for calcul the mask position with numpy : 0.010524272918701172 nb_pixel_total : 3173 time to create 1 rle with old method : 0.0038175582885742188 time for calcul the mask position with numpy : 0.008813858032226562 nb_pixel_total : 34 time to create 1 rle with old method : 4.935264587402344e-05 time for calcul the mask position with numpy : 0.008752107620239258 nb_pixel_total : 67 time to create 1 rle with old method : 9.226799011230469e-05 time for calcul the mask position with numpy : 0.00852513313293457 nb_pixel_total : 961 time to create 1 rle with old method : 0.0011260509490966797 time for calcul the mask position with numpy : 0.00868844985961914 nb_pixel_total : 391 time to create 1 rle with old method : 0.0005085468292236328 time for calcul the mask position with numpy : 0.008630514144897461 nb_pixel_total : 153 time to create 1 rle with old method : 0.0001900196075439453 time for calcul the mask position with numpy : 0.008618593215942383 nb_pixel_total : 1083 time to create 1 rle with old method : 0.0012850761413574219 time for calcul the mask position with numpy : 0.008460044860839844 nb_pixel_total : 3016 time to create 1 rle with old method : 0.003422975540161133 time for calcul the mask position with numpy : 0.008429765701293945 nb_pixel_total : 2943 time to create 1 rle with old method : 0.0033948421478271484 time for calcul the mask position with numpy : 0.008648157119750977 nb_pixel_total : 645 time to create 1 rle with old method : 0.0007410049438476562 time for calcul the mask position with numpy : 0.008895158767700195 nb_pixel_total : 7045 time to create 1 rle with old method : 0.008208751678466797 time for calcul the mask position with numpy : 0.008894920349121094 nb_pixel_total : 10254 time to create 1 rle with old method : 0.012028932571411133 time for calcul the mask position with numpy : 0.008687734603881836 nb_pixel_total : 229 time to create 1 rle with old method : 0.00028061866760253906 time for calcul the mask position with numpy : 0.008648395538330078 nb_pixel_total : 5968 time to create 1 rle with old method : 0.006994962692260742 time for calcul the mask position with numpy : 0.008585453033447266 nb_pixel_total : 1783 time to create 1 rle with old method : 0.002027750015258789 time for calcul the mask position with numpy : 0.008535385131835938 nb_pixel_total : 9 time to create 1 rle with old method : 0.0001201629638671875 time for calcul the mask position with numpy : 0.008448123931884766 nb_pixel_total : 135 time to create 1 rle with old method : 0.00017118453979492188 time for calcul the mask position with numpy : 0.008414506912231445 nb_pixel_total : 576 time to create 1 rle with old method : 0.0007939338684082031 time for calcul the mask position with numpy : 0.008490324020385742 nb_pixel_total : 2108 time to create 1 rle with old method : 0.0024518966674804688 time for calcul the mask position with numpy : 0.008649110794067383 nb_pixel_total : 6328 time to create 1 rle with old method : 0.0072002410888671875 time for calcul the mask position with numpy : 0.008839845657348633 nb_pixel_total : 3444 time to create 1 rle with old method : 0.0040166378021240234 time for calcul the mask position with numpy : 0.008713722229003906 nb_pixel_total : 1651 time to create 1 rle with old method : 0.0018954277038574219 time for calcul the mask position with numpy : 0.00867462158203125 nb_pixel_total : 1232 time to create 1 rle with old method : 0.0014843940734863281 time for calcul the mask position with numpy : 0.008504152297973633 nb_pixel_total : 137 time to create 1 rle with old method : 0.000179290771484375 time for calcul the mask position with numpy : 0.00841665267944336 nb_pixel_total : 8 time to create 1 rle with old method : 2.4318695068359375e-05 time for calcul the mask position with numpy : 0.008473634719848633 nb_pixel_total : 222 time to create 1 rle with old method : 0.0002651214599609375 time for calcul the mask position with numpy : 0.008576393127441406 nb_pixel_total : 2310 time to create 1 rle with old method : 0.002693653106689453 time for calcul the mask position with numpy : 0.00854039192199707 nb_pixel_total : 103 time to create 1 rle with old method : 0.00013446807861328125 time for calcul the mask position with numpy : 0.00875401496887207 nb_pixel_total : 10 time to create 1 rle with old method : 5.53131103515625e-05 time for calcul the mask position with numpy : 0.008620500564575195 nb_pixel_total : 60 time to create 1 rle with old method : 8.130073547363281e-05 time for calcul the mask position with numpy : 0.008554458618164062 nb_pixel_total : 3168 time to create 1 rle with old method : 0.0036211013793945312 time for calcul the mask position with numpy : 0.008530616760253906 nb_pixel_total : 179 time to create 1 rle with old method : 0.000217437744140625 time for calcul the mask position with numpy : 0.008770465850830078 nb_pixel_total : 193 time to create 1 rle with old method : 0.0002377033233642578 time for calcul the mask position with numpy : 0.00873422622680664 nb_pixel_total : 1707 time to create 1 rle with old method : 0.0019502639770507812 time for calcul the mask position with numpy : 0.008582353591918945 nb_pixel_total : 290 time to create 1 rle with old method : 0.0003495216369628906 time for calcul the mask position with numpy : 0.008640527725219727 nb_pixel_total : 125 time to create 1 rle with old method : 0.0001609325408935547 time for calcul the mask position with numpy : 0.008498191833496094 nb_pixel_total : 1096 time to create 1 rle with old method : 0.0016658306121826172 time for calcul the mask position with numpy : 0.009576559066772461 nb_pixel_total : 103699 time to create 1 rle with old method : 0.12271714210510254 time for calcul the mask position with numpy : 0.00908660888671875 nb_pixel_total : 2354 time to create 1 rle with old method : 0.002828836441040039 time for calcul the mask position with numpy : 0.008872509002685547 nb_pixel_total : 252 time to create 1 rle with old method : 0.0003590583801269531 time for calcul the mask position with numpy : 0.008938074111938477 nb_pixel_total : 49 time to create 1 rle with old method : 0.00014281272888183594 time for calcul the mask position with numpy : 0.009871721267700195 nb_pixel_total : 318 time to create 1 rle with old method : 0.0005707740783691406 time for calcul the mask position with numpy : 0.009862184524536133 nb_pixel_total : 145 time to create 1 rle with old method : 0.0002608299255371094 time for calcul the mask position with numpy : 0.009937763214111328 nb_pixel_total : 808 time to create 1 rle with old method : 0.0013663768768310547 time for calcul the mask position with numpy : 0.00999760627746582 nb_pixel_total : 409 time to create 1 rle with old method : 0.00067901611328125 time for calcul the mask position with numpy : 0.009827852249145508 nb_pixel_total : 353 time to create 1 rle with old method : 0.0006542205810546875 time for calcul the mask position with numpy : 0.01008915901184082 nb_pixel_total : 148 time to create 1 rle with old method : 0.0002608299255371094 time for calcul the mask position with numpy : 0.009869813919067383 nb_pixel_total : 73 time to create 1 rle with old method : 0.00015282630920410156 time for calcul the mask position with numpy : 0.00896906852722168 nb_pixel_total : 121 time to create 1 rle with old method : 0.0001678466796875 time for calcul the mask position with numpy : 0.009042024612426758 nb_pixel_total : 86 time to create 1 rle with old method : 0.0001327991485595703 time for calcul the mask position with numpy : 0.008588552474975586 nb_pixel_total : 187 time to create 1 rle with old method : 0.00025153160095214844 time for calcul the mask position with numpy : 0.008522987365722656 nb_pixel_total : 262 time to create 1 rle with old method : 0.0003314018249511719 time for calcul the mask position with numpy : 0.008742809295654297 nb_pixel_total : 208 time to create 1 rle with old method : 0.00032329559326171875 time for calcul the mask position with numpy : 0.009871721267700195 nb_pixel_total : 168 time to create 1 rle with old method : 0.0002694129943847656 time for calcul the mask position with numpy : 0.008791446685791016 nb_pixel_total : 52 time to create 1 rle with old method : 0.00013375282287597656 time for calcul the mask position with numpy : 0.008616924285888672 nb_pixel_total : 117 time to create 1 rle with old method : 0.0001461505889892578 time for calcul the mask position with numpy : 0.0093841552734375 nb_pixel_total : 708 time to create 1 rle with old method : 0.0009336471557617188 time for calcul the mask position with numpy : 0.008509635925292969 nb_pixel_total : 432 time to create 1 rle with old method : 0.0004942417144775391 time for calcul the mask position with numpy : 0.008554220199584961 nb_pixel_total : 383 time to create 1 rle with old method : 0.0004589557647705078 time for calcul the mask position with numpy : 0.008533477783203125 nb_pixel_total : 318 time to create 1 rle with old method : 0.0003914833068847656 time for calcul the mask position with numpy : 0.008617639541625977 nb_pixel_total : 124 time to create 1 rle with old method : 0.00016355514526367188 time for calcul the mask position with numpy : 0.008571147918701172 nb_pixel_total : 763 time to create 1 rle with old method : 0.0008971691131591797 time for calcul the mask position with numpy : 0.008597135543823242 nb_pixel_total : 70 time to create 1 rle with old method : 0.00011324882507324219 time for calcul the mask position with numpy : 0.008742809295654297 nb_pixel_total : 413 time to create 1 rle with old method : 0.0004930496215820312 time for calcul the mask position with numpy : 0.008654117584228516 nb_pixel_total : 421 time to create 1 rle with old method : 0.0005266666412353516 time for calcul the mask position with numpy : 0.008697032928466797 nb_pixel_total : 496 time to create 1 rle with old method : 0.0005946159362792969 time for calcul the mask position with numpy : 0.008650541305541992 nb_pixel_total : 285 time to create 1 rle with old method : 0.0003459453582763672 time for calcul the mask position with numpy : 0.008698225021362305 nb_pixel_total : 998 time to create 1 rle with old method : 0.0012590885162353516 time for calcul the mask position with numpy : 0.00855398178100586 nb_pixel_total : 99 time to create 1 rle with old method : 0.00013971328735351562 time for calcul the mask position with numpy : 0.00858306884765625 nb_pixel_total : 1122 time to create 1 rle with old method : 0.0013103485107421875 time for calcul the mask position with numpy : 0.008539915084838867 nb_pixel_total : 543 time to create 1 rle with old method : 0.00064849853515625 time for calcul the mask position with numpy : 0.008609294891357422 nb_pixel_total : 66 time to create 1 rle with old method : 0.00011348724365234375 time for calcul the mask position with numpy : 0.008589744567871094 nb_pixel_total : 664 time to create 1 rle with old method : 0.0007748603820800781 time for calcul the mask position with numpy : 0.008552789688110352 nb_pixel_total : 640 time to create 1 rle with old method : 0.0008337497711181641 time for calcul the mask position with numpy : 0.008571863174438477 nb_pixel_total : 87 time to create 1 rle with old method : 0.0001266002655029297 time for calcul the mask position with numpy : 0.008568525314331055 nb_pixel_total : 895 time to create 1 rle with old method : 0.0010523796081542969 time for calcul the mask position with numpy : 0.00864100456237793 nb_pixel_total : 342 time to create 1 rle with old method : 0.0004291534423828125 time for calcul the mask position with numpy : 0.008643150329589844 nb_pixel_total : 56 time to create 1 rle with old method : 9.202957153320312e-05 time for calcul the mask position with numpy : 0.008699655532836914 nb_pixel_total : 167 time to create 1 rle with old method : 0.00023937225341796875 time for calcul the mask position with numpy : 0.013090133666992188 nb_pixel_total : 281 time to create 1 rle with old method : 0.00048661231994628906 time for calcul the mask position with numpy : 0.010825157165527344 nb_pixel_total : 31 time to create 1 rle with old method : 9.274482727050781e-05 time for calcul the mask position with numpy : 0.009341001510620117 nb_pixel_total : 197 time to create 1 rle with old method : 0.0002562999725341797 time for calcul the mask position with numpy : 0.008901596069335938 nb_pixel_total : 80 time to create 1 rle with old method : 0.0001289844512939453 time for calcul the mask position with numpy : 0.008797645568847656 nb_pixel_total : 920 time to create 1 rle with old method : 0.0011556148529052734 time for calcul the mask position with numpy : 0.00872349739074707 nb_pixel_total : 108 time to create 1 rle with old method : 0.00015211105346679688 time for calcul the mask position with numpy : 0.00871896743774414 nb_pixel_total : 676 time to create 1 rle with old method : 0.0007863044738769531 time for calcul the mask position with numpy : 0.008590221405029297 nb_pixel_total : 536 time to create 1 rle with old method : 0.0006361007690429688 time for calcul the mask position with numpy : 0.008659601211547852 nb_pixel_total : 296 time to create 1 rle with old method : 0.00035452842712402344 time for calcul the mask position with numpy : 0.008925199508666992 nb_pixel_total : 58 time to create 1 rle with old method : 8.034706115722656e-05 create new chi : 1.6057446002960205 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.02350759506225586 batch 1 Loaded 124 chid ids of type : 4230 Number RLEs to save : 1692 TO DO : save crop sub photo not yet done ! save time : 0.34082794189453125 nb_obj : 122 nb_hashtags : 6 time to prepare the origin masks : 1.2091119289398193 time for calcul the mask position with numpy : 0.06830501556396484 nb_pixel_total : 1814269 time to create 1 rle with new method : 0.5817246437072754 time for calcul the mask position with numpy : 0.008593559265136719 nb_pixel_total : 5249 time to create 1 rle with old method : 0.006005287170410156 time for calcul the mask position with numpy : 0.008822441101074219 nb_pixel_total : 61720 time to create 1 rle with old method : 0.06895041465759277 time for calcul the mask position with numpy : 0.008547782897949219 nb_pixel_total : 1080 time to create 1 rle with old method : 0.001241922378540039 time for calcul the mask position with numpy : 0.008640766143798828 nb_pixel_total : 98 time to create 1 rle with old method : 0.00016427040100097656 time for calcul the mask position with numpy : 0.00857234001159668 nb_pixel_total : 631 time to create 1 rle with old method : 0.0007493495941162109 time for calcul the mask position with numpy : 0.00871586799621582 nb_pixel_total : 37 time to create 1 rle with old method : 7.009506225585938e-05 time for calcul the mask position with numpy : 0.00868678092956543 nb_pixel_total : 100 time to create 1 rle with old method : 0.00018596649169921875 time for calcul the mask position with numpy : 0.008732795715332031 nb_pixel_total : 58 time to create 1 rle with old method : 0.00010180473327636719 time for calcul the mask position with numpy : 0.009360074996948242 nb_pixel_total : 2208 time to create 1 rle with old method : 0.00366973876953125 time for calcul the mask position with numpy : 0.010532379150390625 nb_pixel_total : 133 time to create 1 rle with old method : 0.0002110004425048828 time for calcul the mask position with numpy : 0.008584260940551758 nb_pixel_total : 276 time to create 1 rle with old method : 0.00044465065002441406 time for calcul the mask position with numpy : 0.008699178695678711 nb_pixel_total : 5590 time to create 1 rle with old method : 0.006731271743774414 time for calcul the mask position with numpy : 0.008764982223510742 nb_pixel_total : 3419 time to create 1 rle with old method : 0.004049062728881836 time for calcul the mask position with numpy : 0.008603811264038086 nb_pixel_total : 4 time to create 1 rle with old method : 0.00014162063598632812 time for calcul the mask position with numpy : 0.008563995361328125 nb_pixel_total : 662 time to create 1 rle with old method : 0.0007908344268798828 time for calcul the mask position with numpy : 0.008633852005004883 nb_pixel_total : 1455 time to create 1 rle with old method : 0.0016858577728271484 time for calcul the mask position with numpy : 0.008575677871704102 nb_pixel_total : 379 time to create 1 rle with old method : 0.00047707557678222656 time for calcul the mask position with numpy : 0.008566141128540039 nb_pixel_total : 239 time to create 1 rle with old method : 0.0002899169921875 time for calcul the mask position with numpy : 0.008543729782104492 nb_pixel_total : 870 time to create 1 rle with old method : 0.0010082721710205078 time for calcul the mask position with numpy : 0.008532047271728516 nb_pixel_total : 967 time to create 1 rle with old method : 0.0011179447174072266 time for calcul the mask position with numpy : 0.00849294662475586 nb_pixel_total : 559 time to create 1 rle with old method : 0.0006649494171142578 time for calcul the mask position with numpy : 0.008513927459716797 nb_pixel_total : 1632 time to create 1 rle with old method : 0.0018589496612548828 time for calcul the mask position with numpy : 0.008472442626953125 nb_pixel_total : 2582 time to create 1 rle with old method : 0.002979278564453125 time for calcul the mask position with numpy : 0.008498430252075195 nb_pixel_total : 444 time to create 1 rle with old method : 0.000514984130859375 time for calcul the mask position with numpy : 0.008417844772338867 nb_pixel_total : 5810 time to create 1 rle with old method : 0.006539106369018555 time for calcul the mask position with numpy : 0.00855398178100586 nb_pixel_total : 671 time to create 1 rle with old method : 0.0007805824279785156 time for calcul the mask position with numpy : 0.008504152297973633 nb_pixel_total : 1312 time to create 1 rle with old method : 0.0015366077423095703 time for calcul the mask position with numpy : 0.008518457412719727 nb_pixel_total : 8966 time to create 1 rle with old method : 0.010178565979003906 time for calcul the mask position with numpy : 0.008591413497924805 nb_pixel_total : 212 time to create 1 rle with old method : 0.00026535987854003906 time for calcul the mask position with numpy : 0.00836324691772461 nb_pixel_total : 5800 time to create 1 rle with old method : 0.006722927093505859 time for calcul the mask position with numpy : 0.008541107177734375 nb_pixel_total : 33 time to create 1 rle with old method : 6.031990051269531e-05 time for calcul the mask position with numpy : 0.008519649505615234 nb_pixel_total : 211 time to create 1 rle with old method : 0.00031185150146484375 time for calcul the mask position with numpy : 0.008711099624633789 nb_pixel_total : 123 time to create 1 rle with old method : 0.0001804828643798828 time for calcul the mask position with numpy : 0.008538246154785156 nb_pixel_total : 545 time to create 1 rle with old method : 0.0039577484130859375 time for calcul the mask position with numpy : 0.008716583251953125 nb_pixel_total : 464 time to create 1 rle with old method : 0.0006463527679443359 time for calcul the mask position with numpy : 0.008603096008300781 nb_pixel_total : 2444 time to create 1 rle with old method : 0.0028264522552490234 time for calcul the mask position with numpy : 0.008578777313232422 nb_pixel_total : 1640 time to create 1 rle with old method : 0.0019218921661376953 time for calcul the mask position with numpy : 0.008458137512207031 nb_pixel_total : 3615 time to create 1 rle with old method : 0.004336118698120117 time for calcul the mask position with numpy : 0.008602142333984375 nb_pixel_total : 163 time to create 1 rle with old method : 0.000286102294921875 time for calcul the mask position with numpy : 0.008720636367797852 nb_pixel_total : 661 time to create 1 rle with old method : 0.0008332729339599609 time for calcul the mask position with numpy : 0.00925445556640625 nb_pixel_total : 549 time to create 1 rle with old method : 0.0006542205810546875 time for calcul the mask position with numpy : 0.008644819259643555 nb_pixel_total : 1635 time to create 1 rle with old method : 0.0018908977508544922 time for calcul the mask position with numpy : 0.008524894714355469 nb_pixel_total : 215 time to create 1 rle with old method : 0.00029730796813964844 time for calcul the mask position with numpy : 0.00867009162902832 nb_pixel_total : 180 time to create 1 rle with old method : 0.0002491474151611328 time for calcul the mask position with numpy : 0.008695602416992188 nb_pixel_total : 2227 time to create 1 rle with old method : 0.0025892257690429688 time for calcul the mask position with numpy : 0.008626937866210938 nb_pixel_total : 307 time to create 1 rle with old method : 0.0004076957702636719 time for calcul the mask position with numpy : 0.008620738983154297 nb_pixel_total : 92 time to create 1 rle with old method : 0.00013256072998046875 time for calcul the mask position with numpy : 0.008574724197387695 nb_pixel_total : 65 time to create 1 rle with old method : 0.00011563301086425781 time for calcul the mask position with numpy : 0.008605003356933594 nb_pixel_total : 3116 time to create 1 rle with old method : 0.0036067962646484375 time for calcul the mask position with numpy : 0.00864267349243164 nb_pixel_total : 145 time to create 1 rle with old method : 0.0002040863037109375 time for calcul the mask position with numpy : 0.008610248565673828 nb_pixel_total : 1758 time to create 1 rle with old method : 0.002084970474243164 time for calcul the mask position with numpy : 0.00836944580078125 nb_pixel_total : 213 time to create 1 rle with old method : 0.0003135204315185547 time for calcul the mask position with numpy : 0.008476018905639648 nb_pixel_total : 322 time to create 1 rle with old method : 0.00036787986755371094 time for calcul the mask position with numpy : 0.008412361145019531 nb_pixel_total : 1090 time to create 1 rle with old method : 0.0012459754943847656 time for calcul the mask position with numpy : 0.008537769317626953 nb_pixel_total : 120 time to create 1 rle with old method : 0.0001678466796875 time for calcul the mask position with numpy : 0.008492708206176758 nb_pixel_total : 100 time to create 1 rle with old method : 0.00014829635620117188 time for calcul the mask position with numpy : 0.009070873260498047 nb_pixel_total : 105074 time to create 1 rle with old method : 0.13233518600463867 time for calcul the mask position with numpy : 0.009802579879760742 nb_pixel_total : 273 time to create 1 rle with old method : 0.000518798828125 time for calcul the mask position with numpy : 0.009837150573730469 nb_pixel_total : 30 time to create 1 rle with old method : 0.00011897087097167969 time for calcul the mask position with numpy : 0.009760856628417969 nb_pixel_total : 2454 time to create 1 rle with old method : 0.0030164718627929688 time for calcul the mask position with numpy : 0.008638858795166016 nb_pixel_total : 152 time to create 1 rle with old method : 0.000202178955078125 time for calcul the mask position with numpy : 0.008568048477172852 nb_pixel_total : 277 time to create 1 rle with old method : 0.00035262107849121094 time for calcul the mask position with numpy : 0.008637189865112305 nb_pixel_total : 120 time to create 1 rle with old method : 0.00017523765563964844 time for calcul the mask position with numpy : 0.008696317672729492 nb_pixel_total : 4 time to create 1 rle with old method : 2.9087066650390625e-05 time for calcul the mask position with numpy : 0.00861668586730957 nb_pixel_total : 289 time to create 1 rle with old method : 0.00037360191345214844 time for calcul the mask position with numpy : 0.008633136749267578 nb_pixel_total : 886 time to create 1 rle with old method : 0.0010457038879394531 time for calcul the mask position with numpy : 0.008649110794067383 nb_pixel_total : 149 time to create 1 rle with old method : 0.00019621849060058594 time for calcul the mask position with numpy : 0.00862431526184082 nb_pixel_total : 343 time to create 1 rle with old method : 0.00046944618225097656 time for calcul the mask position with numpy : 0.008608818054199219 nb_pixel_total : 617 time to create 1 rle with old method : 0.0007314682006835938 time for calcul the mask position with numpy : 0.008571863174438477 nb_pixel_total : 89 time to create 1 rle with old method : 0.00012159347534179688 time for calcul the mask position with numpy : 0.00850534439086914 nb_pixel_total : 116 time to create 1 rle with old method : 0.00015473365783691406 time for calcul the mask position with numpy : 0.008478879928588867 nb_pixel_total : 1 time to create 1 rle with old method : 1.52587890625e-05 time for calcul the mask position with numpy : 0.008478641510009766 nb_pixel_total : 185 time to create 1 rle with old method : 0.00022721290588378906 time for calcul the mask position with numpy : 0.008474588394165039 nb_pixel_total : 351 time to create 1 rle with old method : 0.00042939186096191406 time for calcul the mask position with numpy : 0.008523225784301758 nb_pixel_total : 480 time to create 1 rle with old method : 0.0005543231964111328 time for calcul the mask position with numpy : 0.008482694625854492 nb_pixel_total : 304 time to create 1 rle with old method : 0.0003609657287597656 time for calcul the mask position with numpy : 0.008462905883789062 nb_pixel_total : 11 time to create 1 rle with old method : 3.457069396972656e-05 time for calcul the mask position with numpy : 0.00846242904663086 nb_pixel_total : 306 time to create 1 rle with old method : 0.000362396240234375 time for calcul the mask position with numpy : 0.008485555648803711 nb_pixel_total : 14 time to create 1 rle with old method : 4.696846008300781e-05 time for calcul the mask position with numpy : 0.008428573608398438 nb_pixel_total : 22 time to create 1 rle with old method : 5.936622619628906e-05 time for calcul the mask position with numpy : 0.008487224578857422 nb_pixel_total : 411 time to create 1 rle with old method : 0.0005295276641845703 time for calcul the mask position with numpy : 0.008495092391967773 nb_pixel_total : 118 time to create 1 rle with old method : 0.00015974044799804688 time for calcul the mask position with numpy : 0.008554697036743164 nb_pixel_total : 723 time to create 1 rle with old method : 0.0008766651153564453 time for calcul the mask position with numpy : 0.008611917495727539 nb_pixel_total : 406 time to create 1 rle with old method : 0.0004894733428955078 time for calcul the mask position with numpy : 0.008460283279418945 nb_pixel_total : 400 time to create 1 rle with old method : 0.0005197525024414062 time for calcul the mask position with numpy : 0.008461475372314453 nb_pixel_total : 74 time to create 1 rle with old method : 0.00018835067749023438 time for calcul the mask position with numpy : 0.008449077606201172 nb_pixel_total : 308 time to create 1 rle with old method : 0.0003902912139892578 time for calcul the mask position with numpy : 0.008513689041137695 nb_pixel_total : 154 time to create 1 rle with old method : 0.00021266937255859375 time for calcul the mask position with numpy : 0.008583307266235352 nb_pixel_total : 8 time to create 1 rle with old method : 5.53131103515625e-05 time for calcul the mask position with numpy : 0.008578062057495117 nb_pixel_total : 346 time to create 1 rle with old method : 0.0004360675811767578 time for calcul the mask position with numpy : 0.00853109359741211 nb_pixel_total : 1025 time to create 1 rle with old method : 0.001196146011352539 time for calcul the mask position with numpy : 0.008550643920898438 nb_pixel_total : 29 time to create 1 rle with old method : 5.5789947509765625e-05 time for calcul the mask position with numpy : 0.008549690246582031 nb_pixel_total : 240 time to create 1 rle with old method : 0.00030112266540527344 time for calcul the mask position with numpy : 0.008576393127441406 nb_pixel_total : 819 time to create 1 rle with old method : 0.0010826587677001953 time for calcul the mask position with numpy : 0.008629322052001953 nb_pixel_total : 90 time to create 1 rle with old method : 0.0002040863037109375 time for calcul the mask position with numpy : 0.008577823638916016 nb_pixel_total : 15 time to create 1 rle with old method : 4.887580871582031e-05 time for calcul the mask position with numpy : 0.00857853889465332 nb_pixel_total : 56 time to create 1 rle with old method : 9.417533874511719e-05 time for calcul the mask position with numpy : 0.008581161499023438 nb_pixel_total : 3 time to create 1 rle with old method : 3.123283386230469e-05 time for calcul the mask position with numpy : 0.008584737777709961 nb_pixel_total : 72 time to create 1 rle with old method : 0.00011730194091796875 time for calcul the mask position with numpy : 0.008653402328491211 nb_pixel_total : 887 time to create 1 rle with old method : 0.0010759830474853516 time for calcul the mask position with numpy : 0.008570194244384766 nb_pixel_total : 426 time to create 1 rle with old method : 0.0005092620849609375 time for calcul the mask position with numpy : 0.008574724197387695 nb_pixel_total : 57 time to create 1 rle with old method : 9.894371032714844e-05 time for calcul the mask position with numpy : 0.008523941040039062 nb_pixel_total : 51 time to create 1 rle with old method : 0.00010585784912109375 time for calcul the mask position with numpy : 0.008655786514282227 nb_pixel_total : 684 time to create 1 rle with old method : 0.0008077621459960938 time for calcul the mask position with numpy : 0.00863337516784668 nb_pixel_total : 635 time to create 1 rle with old method : 0.0007851123809814453 time for calcul the mask position with numpy : 0.008634328842163086 nb_pixel_total : 48 time to create 1 rle with old method : 9.822845458984375e-05 time for calcul the mask position with numpy : 0.008570671081542969 nb_pixel_total : 82 time to create 1 rle with old method : 0.00011444091796875 time for calcul the mask position with numpy : 0.00859212875366211 nb_pixel_total : 251 time to create 1 rle with old method : 0.0003056526184082031 time for calcul the mask position with numpy : 0.008562564849853516 nb_pixel_total : 6 time to create 1 rle with old method : 3.8623809814453125e-05 time for calcul the mask position with numpy : 0.00858163833618164 nb_pixel_total : 7 time to create 1 rle with old method : 8.821487426757812e-05 time for calcul the mask position with numpy : 0.008574962615966797 nb_pixel_total : 52 time to create 1 rle with old method : 0.00010824203491210938 time for calcul the mask position with numpy : 0.008528947830200195 nb_pixel_total : 342 time to create 1 rle with old method : 0.0004200935363769531 time for calcul the mask position with numpy : 0.008535146713256836 nb_pixel_total : 47 time to create 1 rle with old method : 8.296966552734375e-05 time for calcul the mask position with numpy : 0.008485555648803711 nb_pixel_total : 4 time to create 1 rle with old method : 2.5510787963867188e-05 time for calcul the mask position with numpy : 0.008461236953735352 nb_pixel_total : 311 time to create 1 rle with old method : 0.0003650188446044922 time for calcul the mask position with numpy : 0.008451461791992188 nb_pixel_total : 7 time to create 1 rle with old method : 6.246566772460938e-05 time for calcul the mask position with numpy : 0.008434295654296875 nb_pixel_total : 819 time to create 1 rle with old method : 0.0009548664093017578 time for calcul the mask position with numpy : 0.008461952209472656 nb_pixel_total : 672 time to create 1 rle with old method : 0.0008003711700439453 time for calcul the mask position with numpy : 0.008536100387573242 nb_pixel_total : 526 time to create 1 rle with old method : 0.0006434917449951172 time for calcul the mask position with numpy : 0.008527278900146484 nb_pixel_total : 291 time to create 1 rle with old method : 0.0003514289855957031 time for calcul the mask position with numpy : 0.008594036102294922 nb_pixel_total : 33 time to create 1 rle with old method : 0.00012373924255371094 time for calcul the mask position with numpy : 0.008551359176635742 nb_pixel_total : 53 time to create 1 rle with old method : 8.7738037109375e-05 create new chi : 2.0347652435302734 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.0016276836395263672 batch 1 Loaded 125 chid ids of type : 4230 Number RLEs to save : 757 TO DO : save crop sub photo not yet done ! save time : 0.64597487449646 map_output_result : {1342560035: (0.0, 'Should be the crop_list due to order', 0.0), 1342560015: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1342560035, 1342560015] Looping around the photos to save general results len do output : 2 /1342560035.Didn't retrieve data . /1342560015.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2654697') ('4234', '21225802', '1342560035', None, None, None, None, None, '2654697') ('4234', None, None, None, None, None, None, None, '2654697') ('4234', '21225802', '1342560015', None, None, None, None, None, '2654697') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.033141136169433594 save_final save missing photos in datou_result : time spend for datou_step_exec : 7.491322994232178 time spend to save output : 0.033509254455566406 total time spend for step 4 : 7.524832248687744 step5:crop_condition Mon Mar 10 16:21:01 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4230 Loading chi in step crop for list_pids : 2 ! batch 1 Loaded 249 chid ids of type : 4230 begin to crop the class : papier param for this class : {'min_score': 0.6} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 87 About to insert : list_path_to_insert length 87 new photo from crops ! About to upload 87 photos upload in portfolio : 4869462 init cache_photo without model_param we have 87 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741620064_3432003 we have uploaded 87 photos in the portfolio 4869462 time of upload the photos Elapsed time : 50.24094772338867 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.6} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 24 About to insert : list_path_to_insert length 24 new photo from crops ! About to upload 24 photos upload in portfolio : 4869462 init cache_photo without model_param we have 24 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741620114_3432003 we have uploaded 24 photos in the portfolio 4869462 time of upload the photos Elapsed time : 9.794613599777222 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.6} filtre for class : metal hashtag_id of this class : 492628673 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741620124_3432003 we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.6881349086761475 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.6} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! About to upload 8 photos upload in portfolio : 4869462 init cache_photo without model_param we have 8 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741620125_3432003 we have uploaded 8 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.8745644092559814 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.6} filtre for class : autre hashtag_id of this class : 494826614 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 34 About to insert : list_path_to_insert length 34 new photo from crops ! About to upload 34 photos upload in portfolio : 4869462 init cache_photo without model_param we have 34 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741620129_3432003 we have uploaded 34 photos in the portfolio 4869462 time of upload the photos Elapsed time : 32.76691675186157 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.6} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.6} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1342560035, 1342560015] Looping around the photos to save general results len do output : 154 /1342567972Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342567973Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342567974Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342567975Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342567976Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342567977Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342567978Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342567979Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342567980Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342567982Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342567983Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342567984Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342567985Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342567986Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342567987Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342567988Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342567989Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342567990Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342567991Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342567992Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342567993Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342567994Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342567995Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342567996Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342567997Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342567998Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342567999Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568000Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568001Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568002Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568003Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568006Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568009Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568012Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568016Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568020Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568024Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568027Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568031Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568034Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568038Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568041Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568045Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568049Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568054Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568057Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568061Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568066Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568070Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568074Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568079Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568083Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568087Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568091Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568095Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568100Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568105Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568110Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568115Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568119Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568123Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568127Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568132Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568136Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568140Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568145Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568149Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568154Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568158Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568163Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568167Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568171Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568175Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568179Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568184Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568189Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568193Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568198Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568206Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568211Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568216Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568220Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568225Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568229Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568234Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568239Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568244Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568299Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568300Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568302Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568303Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568304Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568305Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568306Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568307Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568309Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568310Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568311Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568312Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568313Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568315Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568316Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568317Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568318Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568319Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568320Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568321Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568322Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568323Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568324Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568325Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568327Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568329Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568330Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568331Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568332Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568333Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568334Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568335Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568336Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568391Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568392Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568393Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568394Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568395Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568396Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568397Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568398Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568400Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568401Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568403Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568404Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568406Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568407Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568408Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568409Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568410Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568411Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568412Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568414Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568415Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568416Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568417Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568418Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568419Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568420Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568421Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568422Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568423Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568424Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568425Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568426Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568429Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342568430Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2654697') ('4234', '21225802', '1342560035', None, None, None, None, None, '2654697') ('4234', None, None, None, None, None, None, None, '2654697') ('4234', '21225802', '1342560015', None, None, None, None, None, '2654697') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 464 time used for this insertion : 0.0561826229095459 save_final save missing photos in datou_result : time spend for datou_step_exec : 100.17373704910278 time spend to save output : 0.060906410217285156 total time spend for step 5 : 100.23464345932007 step6:thcl Mon Mar 10 16:22:41 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! we are using the classfication for only one thcl 3237 time to import caffe and check if the image exist : 0.017943859100341797 time to convert the images to numpy array : 0.0258481502532959 time to import caffe and check if the image exist : 0.01607060432434082 time to convert the images to numpy array : 0.03442502021789551 time to import caffe and check if the image exist : 0.013056516647338867 time to convert the images to numpy array : 0.04177737236022949 time to import caffe and check if the image exist : 0.01245737075805664 time to convert the images to numpy array : 0.04772782325744629 time to import caffe and check if the image exist : 0.02643585205078125 time to convert the images to numpy array : 0.04453539848327637 time to import caffe and check if the image exist : 0.02165508270263672 time to convert the images to numpy array : 0.05028676986694336 time to import caffe and check if the image exist : 0.024141550064086914 time to convert the images to numpy array : 0.047422170639038086 time to import caffe and check if the image exist : 0.0238189697265625 time to convert the images to numpy array : 0.04797673225402832 time to import caffe and check if the image exist : 0.03621625900268555 time to convert the images to numpy array : 0.036765098571777344 time to import caffe and check if the image exist : 0.03036212921142578 time to convert the images to numpy array : 0.04224228858947754 total time to convert the images to numpy array : 0.40955257415771484 list photo_ids error: [] list photo_ids correct : [1342568310, 1342568311, 1342568312, 1342568313, 1342568315, 1342568316, 1342568317, 1342568318, 1342568319, 1342568320, 1342568321, 1342568322, 1342568323, 1342568324, 1342568325, 1342568327, 1342568419, 1342568420, 1342568421, 1342568422, 1342568423, 1342568424, 1342568425, 1342568426, 1342568429, 1342568430, 1342568400, 1342568401, 1342568403, 1342568404, 1342568406, 1342568407, 1342568408, 1342568409, 1342568410, 1342568411, 1342568412, 1342568414, 1342568415, 1342568416, 1342568417, 1342568418, 1342568329, 1342568330, 1342568331, 1342568332, 1342568333, 1342568334, 1342568335, 1342568336, 1342568391, 1342568392, 1342568393, 1342568394, 1342568395, 1342568396, 1342568397, 1342568398, 1342568070, 1342568074, 1342568079, 1342568083, 1342568087, 1342568091, 1342568095, 1342568100, 1342568105, 1342568110, 1342568115, 1342568119, 1342568123, 1342568127, 1342568132, 1342568136, 1342567972, 1342567973, 1342567974, 1342567975, 1342567976, 1342567977, 1342567978, 1342567979, 1342567980, 1342567982, 1342567983, 1342567984, 1342567985, 1342567986, 1342567987, 1342567988, 1342568140, 1342568145, 1342568149, 1342568154, 1342568158, 1342568163, 1342568167, 1342568171, 1342568175, 1342568179, 1342568184, 1342568189, 1342568193, 1342568198, 1342568206, 1342568211, 1342568216, 1342568220, 1342568225, 1342568229, 1342568234, 1342568239, 1342568244, 1342568299, 1342568300, 1342568302, 1342568303, 1342568304, 1342568305, 1342568306, 1342568307, 1342568309, 1342567989, 1342567990, 1342567991, 1342567992, 1342567993, 1342567994, 1342567995, 1342567996, 1342567997, 1342567998, 1342567999, 1342568000, 1342568001, 1342568002, 1342568003, 1342568006, 1342568009, 1342568012, 1342568016, 1342568020, 1342568024, 1342568027, 1342568031, 1342568034, 1342568038, 1342568041, 1342568045, 1342568049, 1342568054, 1342568057, 1342568061, 1342568066] number of photos to traite : 154 try to delete the photos incorrect in DB tagging for thcl : 3237 To do loadFromThcl(), then load ParamDescType : thcl3237 thcls : [{'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'}] thcl {'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'} Update svm_hashtag_type_desc : 5561 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) To loadFromThcl() : net_5561 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 9573 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_rubbia_refus_2500 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_rubbia_refus_2500 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_rubbia_refus_2500 /data/models_weight/learn_rubbia_refus_2500/caffemodel size_local : 94358479 size in s3 : 94358479 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:16 caffemodel already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt size_local : 32544 size in s3 : 32544 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:15 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:09:52 mean.npy already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/synset_words.txt size_local : 334 size in s3 : 334 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:10:06 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt caffemodel_filename : /data/models_weight/learn_rubbia_refus_2500/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 9354 max_wait_temp : 1 max_wait : 0 dict_keys(['prob', 'pool5']) time used to do the prepocess of the images : 0.9197769165039062 time used to do the prediction : 0.7758731842041016 save descriptor for thcl : 3237 time to traite the descriptors : 0.9906919002532959 Catched exception ! Connect or reconnect ! storage_type for insertDescriptorsMulti : 3 To insert : 1342568310 To insert : 1342568311 To insert : 1342568312 To insert : 1342568313 To insert : 1342568315 To insert : 1342568316 To insert : 1342568317 To insert : 1342568318 To insert : 1342568319 To insert : 1342568320 To insert : 1342568321 To insert : 1342568322 To insert : 1342568323 To insert : 1342568324 To insert : 1342568325 To insert : 1342568327 To insert : 1342568419 To insert : 1342568420 To insert : 1342568421 To insert : 1342568422 To insert : 1342568423 To insert : 1342568424 To insert : 1342568425 To insert : 1342568426 To insert : 1342568429 To insert : 1342568430 To insert : 1342568400 To insert : 1342568401 To insert : 1342568403 To insert : 1342568404 To insert : 1342568406 To insert : 1342568407 To insert : 1342568408 To insert : 1342568409 To insert : 1342568410 To insert : 1342568411 To insert : 1342568412 To insert : 1342568414 To insert : 1342568415 To insert : 1342568416 To insert : 1342568417 To insert : 1342568418 To insert : 1342568329 To insert : 1342568330 To insert : 1342568331 To insert : 1342568332 To insert : 1342568333 To insert : 1342568334 To insert : 1342568335 To insert : 1342568336 To insert : 1342568391 To insert : 1342568392 To insert : 1342568393 To insert : 1342568394 To insert : 1342568395 To insert : 1342568396 To insert : 1342568397 To insert : 1342568398 To insert : 1342568070 To insert : 1342568074 To insert : 1342568079 To insert : 1342568083 To insert : 1342568087 To insert : 1342568091 To insert : 1342568095 To insert : 1342568100 To insert : 1342568105 To insert : 1342568110 To insert : 1342568115 To insert : 1342568119 To insert : 1342568123 To insert : 1342568127 To insert : 1342568132 To insert : 1342568136 To insert : 1342567972 To insert : 1342567973 To insert : 1342567974 To insert : 1342567975 To insert : 1342567976 To insert : 1342567977 To insert : 1342567978 To insert : 1342567979 To insert : 1342567980 To insert : 1342567982 To insert : 1342567983 To insert : 1342567984 To insert : 1342567985 To insert : 1342567986 To insert : 1342567987 To insert : 1342567988 To insert : 1342568140 To insert : 1342568145 To insert : 1342568149 To insert : 1342568154 To insert : 1342568158 To insert : 1342568163 To insert : 1342568167 To insert : 1342568171 To insert : 1342568175 To insert : 1342568179 To insert : 1342568184 To insert : 1342568189 To insert : 1342568193 To insert : 1342568198 To insert : 1342568206 To insert : 1342568211 To insert : 1342568216 To insert : 1342568220 To insert : 1342568225 To insert : 1342568229 To insert : 1342568234 To insert : 1342568239 To insert : 1342568244 To insert : 1342568299 To insert : 1342568300 To insert : 1342568302 To insert : 1342568303 To insert : 1342568304 To insert : 1342568305 To insert : 1342568306 To insert : 1342568307 To insert : 1342568309 To insert : 1342567989 To insert : 1342567990 To insert : 1342567991 To insert : 1342567992 To insert : 1342567993 To insert : 1342567994 To insert : 1342567995 To insert : 1342567996 To insert : 1342567997 To insert : 1342567998 To insert : 1342567999 To insert : 1342568000 To insert : 1342568001 To insert : 1342568002 To insert : 1342568003 To insert : 1342568006 To insert : 1342568009 To insert : 1342568012 To insert : 1342568016 To insert : 1342568020 To insert : 1342568024 To insert : 1342568027 To insert : 1342568031 To insert : 1342568034 To insert : 1342568038 To insert : 1342568041 To insert : 1342568045 To insert : 1342568049 To insert : 1342568054 To insert : 1342568057 To insert : 1342568061 To insert : 1342568066 time to insert the descriptors : 34.123515129089355 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos SAVE THCL : begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 233 time used for this insertion : 0.04179692268371582 save missing photos in datou_result : time spend for datou_step_exec : 41.61515474319458 time spend to save output : 0.14710092544555664 total time spend for step 6 : 41.76225566864014 step7:ventilate_hashtags_in_portfolio Mon Mar 10 16:23:23 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 21225802 get user id for portfolio 21225802 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21225802 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('refus','pet_clair','autre','metal','carton','papier','Film_plastique','pehd','Carton','PEHD','pet_fonce','environnement','Tetrapak','Papier','PET_clair','mal_croppe','flou','PET_fonce')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21225802 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('refus','pet_clair','autre','metal','carton','papier','Film_plastique','pehd','Carton','PEHD','pet_fonce','environnement','Tetrapak','Papier','PET_clair','mal_croppe','flou','PET_fonce')) AND mptpi.`min_score`=0.6 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21225802 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('refus','pet_clair','autre','metal','carton','papier','Film_plastique','pehd','Carton','PEHD','pet_fonce','environnement','Tetrapak','Papier','PET_clair','mal_croppe','flou','PET_fonce')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21225802 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('refus','pet_clair','autre','metal','carton','papier','Film_plastique','pehd','Carton','PEHD','pet_fonce','environnement','Tetrapak','Papier','PET_clair','mal_croppe','flou','PET_fonce')) AND mptpi.`min_score`=0.6 To do lien utilise dans velours : https://www.fotonower.com/velours/21226973,21226987,21226975,21226976,21226981,21226986,21226979,21226982,21226990,21226984,21226985,21226988,21226989?tags=refus,pet_clair,autre,metal,carton,papier,film_plastique,pehd,pet_fonce,environnement,tetrapak,mal_croppe,flou&datou_id_consolidate=4235&port_consolidate=21225802 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1342560035, 1342560015] Looping around the photos to save general results len do output : 1 /21225802. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2654697') ('4234', '21225802', '1342560035', None, None, None, None, None, '2654697') ('4234', None, None, None, None, None, None, None, '2654697') ('4234', '21225802', '1342560015', None, None, None, None, None, '2654697') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.02661299705505371 save_final save missing photos in datou_result : time spend for datou_step_exec : 5.517609596252441 time spend to save output : 0.026904582977294922 total time spend for step 7 : 5.544514179229736 step8:final Mon Mar 10 16:23:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1342560035: ('0.20266772831754704',), 1342560015: ('0.20266772831754704',)} new output for save of step final : {1342560035: ('0.20266772831754704',), 1342560015: ('0.20266772831754704',)} [1342560035, 1342560015] Looping around the photos to save general results len do output : 2 /1342560035.Didn't retrieve data . /1342560015.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2654697') ('4234', '21225802', '1342560035', None, None, None, None, None, '2654697') ('4234', None, None, None, None, None, None, None, '2654697') ('4234', '21225802', '1342560015', None, None, None, None, None, '2654697') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.029271841049194336 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.38805603981018066 time spend to save output : 0.02952742576599121 total time spend for step 8 : 0.4175834655761719 step9:velours_tree Mon Mar 10 16:23:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure list_portfolios : 21226973,21226987,21226975,21226976,21226981,21226986,21226979,21226982,21226990,21226984,21226985,21226988,21226989 photo desc type : 5561 - Retrieving photos to tag... query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21226973 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21226987 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21226975 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21226976 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21226981 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21226986 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21226979 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21226982 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21226990 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21226984 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21226985 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21226988 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21226989 ORDER BY ph.size desc - Loading descriptors... Size : 2048 len(descriptors) : 154 Compute structured hierarchical clustering... ward : AgglomerativeClustering(n_clusters=6) ward.labels_ : [1 3 2 0 3 2 5 0 2 2 0 2 3 3 2 0 2 2 5 0 2 2 2 3 5 4 2 3 0 3 2 2 3 2 2 0 5 2 2 2 2 5 3 2 1 1 5 1 4 2 3 2 0 2 2 2 2 2 1 3 0 1 0 2 3 0 0 0 5 3 0 3 2 3 2 4 3 0 2 2 3 0 2 2 2 1 0 1 4 0 2 5 0 0 2 0 1 1 0 2 2 2 2 2 3 2 2 3 0 0 0 2 0 0 2 5 2 3 2 0 1 5 0 4 0 0 1 0 2 2 3 0 1 3 0 5 3 1 5 0 4 2 0 2 1 2 3 5 3 0 1 3 0 3] Elapsed time: 0.027541637420654297 graph_id used : 82499 - Beta version, working pretty good on 11-5-16 ! https://www.fotonower.com/velours/21226973,21226987,21226975,21226976,21226981,21226986,21226979,21226982,21226990,21226984,21226985,21226988,21226989?tags=refus,pet_clair,autre,metal,carton,papier,film_plastique,pehd,pet_fonce,environnement,tetrapak,mal_croppe,flou&datou_id_consolidate=4235&port_consolidate=21225802&tree_id=82499 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : velours_tree we use saveGeneral [1342560035, 1342560015] Looping around the photos to save general results len do output : 1 /21225802Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2654697') ('4234', '21225802', '1342560035', None, None, None, None, None, '2654697') ('4234', None, None, None, None, None, None, None, '2654697') ('4234', '21225802', '1342560015', None, None, None, None, None, '2654697') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.0384068489074707 save_final save missing photos in datou_result : time spend for datou_step_exec : 21.841559171676636 time spend to save output : 0.0386652946472168 total time spend for step 9 : 21.880224466323853 step10:send_mail_cod Mon Mar 10 16:23:51 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Qualipapia_P21225802_10-03-2025_16_23_51.pdf 21226955 imagette212269551741620231 21226969 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212269691741620231 21226957 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212269571741620231 21226958 change filename to text .imagette212269581741620232 21226963 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212269631741620233 21226968 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212269681741620233 21226961 imagette212269611741620234 21226964 imagette212269641741620234 21226972 imagette212269721741620234 21226967 imagette212269671741620234 21226970 imagette212269701741620235 21226971 imagette212269711741620235 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=21225802 and hashtag_type = 4230 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/21226973,21226987,21226975,21226976,21226981,21226986,21226979,21226982,21226990,21226984,21226985,21226988,21226989?tags=refus,pet_clair,autre,metal,carton,papier,film_plastique,pehd,pet_fonce,environnement,tetrapak,mal_croppe,flou&datou_id_consolidate=4235&port_consolidate=21225802&tree_id=82499 your option no_mail is active, we will not send the real mail to your client args[1342560035] : ((1342560035, -7.49586047483295, 492609224), (1342560035, -0.6283440731404839, 501862349), '0.20266772831754704') We are sending mail with results at cod@fotonower.com args[1342560015] : ((1342560015, -7.431085668756551, 492609224), (1342560015, -0.6056185531668327, 501862349), '0.20266772831754704') We are sending mail with results at cod@fotonower.com refus_total : 0.20266772831754704 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21225802 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21225802_10-03-2025_16_23_51.pdf results_Qualipapia_P21225802_10-03-2025_16_23_51.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21225802_10-03-2025_16_23_51.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4234','21225802','results_Qualipapia_P21225802_10-03-2025_16_23_51.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21225802_10-03-2025_16_23_51.pdf','pdf','','0.14','0.20266772831754704') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1342560035, 1342560015] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2654697') ('4234', '21225802', '1342560035', None, None, None, None, None, '2654697') ('4234', None, None, None, None, None, None, None, '2654697') ('4234', '21225802', '1342560015', None, None, None, None, None, '2654697') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 2 time used for this insertion : 0.04410290718078613 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.556016206741333 time spend to save output : 0.04425191879272461 total time spend for step 10 : 4.600268125534058 step11:split_time_score Mon Mar 10 16:23:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 3442, 'mtr_user_id': 31, 'name': 'classifieur_2camions_valcor_021122_v1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'deux_camions,camion_droite,camion_gauche,pas_de_camion', 'svm_portfolios_learning': '7659379,7659034,7657685,7657114', 'photo_hashtag_type': 4458, 'photo_desc_type': 5723, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107760533,2107760534,2107760535,2107760536'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('12', 2),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 10032025 21225802 Nombre de photos uploadées : 2 / 23040 (0%) 10032025 21225802 Nombre de photos taguées (types de déchets): 2 / 2 (100%) 10032025 21225802 Nombre de photos taguées (volume) : 0 / 2 (0%) elapsed_time : load_data_split_time_score 2.86102294921875e-06 elapsed_time : order_list_meta_photo_and_scores 9.059906005859375e-06 ?? elapsed_time : fill_and_build_computed_from_old_data 0.0002315044403076172 elapsed_time : insert_dashboard_record_day_entry 0.08838224411010742 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21225722 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21225725 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21225727 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21225729 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21225730 order by id desc limit 1 Qualite : 0.22583614925806508 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21225731_10-03-2025_16_05_27.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21225731 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21225731 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21224373 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21224374 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21224375 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21224376 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21224377 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21224378 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222441 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222442 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222444 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222445 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222446 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222447 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222448 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222449 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222466 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222483 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222484 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222485 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222486 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222487 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222489 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222490 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222491 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222493 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21221311 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21221312 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21221313 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21221314 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21221315 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21221316 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21221317 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21221319 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21221321 order by id desc limit 1 Qualite : 0.21046428553265062 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21221322_10-03-2025_14_57_21.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21221322 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21221322 AND mptpi.`type`=4230 To do Qualite : 0.2463029070088356 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21221123_10-03-2025_14_21_45.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21221123 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21221123 AND mptpi.`type`=4230 To do Qualite : 0.2287247994172867 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21221324_10-03-2025_14_56_23.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21221324 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21221324 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21221326 order by id desc limit 1 Qualite : 0.23017781634605472 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21221327_10-03-2025_14_53_10.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21221327 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21221327 AND mptpi.`type`=4230 To do Qualite : 0.2392766488331622 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21221329_10-03-2025_14_48_34.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21221329 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21221329 AND mptpi.`type`=4230 To do Qualite : 0.2416098495827734 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21221331_10-03-2025_14_47_47.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21221331 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21221331 AND mptpi.`type`=4230 To do Qualite : 0.21151262987772793 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21221332_10-03-2025_14_47_13.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21221332 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21221332 AND mptpi.`type`=4230 To do Qualite : 0.1866215700653093 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21221333_10-03-2025_14_45_45.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21221333 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21221333 AND mptpi.`type`=4230 To do Qualite : 0.2476384769968913 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21221334_10-03-2025_14_47_54.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21221334 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21221334 AND mptpi.`type`=4230 To do Qualite : 0.22266474871535327 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21221335_10-03-2025_14_51_14.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21221335 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21221335 AND mptpi.`type`=4230 To do Qualite : 0.21282245175539322 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21221337_10-03-2025_14_36_14.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21221337 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21221337 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222498 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222501 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222502 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222503 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222504 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222505 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222508 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222511 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222514 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222517 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222519 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222522 order by id desc limit 1 Qualite : 0.213978209235516 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21222525_10-03-2025_15_32_58.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222525 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21222525 AND mptpi.`type`=4230 To do Qualite : 0.2041695679450676 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21222528_10-03-2025_15_07_49.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222528 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21222528 AND mptpi.`type`=4230 To do Qualite : 0.20450723112275113 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21222534_10-03-2025_15_18_40.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222534 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21222534 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222539 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21224380 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21224381 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21224382 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21224383 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21224384 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21224385 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21224386 order by id desc limit 1 Qualite : 0.13829442059756816 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21224387_10-03-2025_16_03_51.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21224387 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21224387 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21224388 order by id desc limit 1 Qualite : 0.19934454301638904 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21224389_10-03-2025_15_51_37.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21224389 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21224389 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21225744 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21225745 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21225746 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21225754 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21225760 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21225761 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21225762 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21225787 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21225788 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21225790 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21225792 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21225793 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21225795 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21225796 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21225798 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21225799 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21225800 order by id desc limit 1 Qualite : 0.20266772831754704 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21225802_10-03-2025_16_23_51.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21225802 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21225802 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21225803 order by id desc limit 1 Qualite : 0.22444304529398856 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21225804_10-03-2025_16_16_54.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21225804 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21225804 AND mptpi.`type`=4230 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'10032025': {'nb_upload': 2, 'nb_taggue_class': 2, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1342560035, 1342560015] Looping around the photos to save general results len do output : 1 /21225802Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2654697') ('4234', '21225802', '1342560035', None, None, None, None, None, '2654697') ('4234', None, None, None, None, None, None, None, '2654697') ('4234', '21225802', '1342560015', None, None, None, None, None, '2654697') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.02763652801513672 save_final save missing photos in datou_result : time spend for datou_step_exec : 72.19459748268127 time spend to save output : 0.02788853645324707 total time spend for step 11 : 72.22248601913452 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 1 len(input) + len(total_photo_id_missing) : 2 set_done_treatment 33.13user 21.83system 4:44.30elapsed 19%CPU (0avgtext+0avgdata 2476364maxresident)k 924696inputs+11384outputs (1972major+2115053minor)pagefaults 0swaps