python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 4234 -l 5 ' -s datou_current_4234 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 906067 load datou : 4234 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load THCL from format json or kwargs add thcl : 3237 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig add pdt : 5561 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4234, datou_cur_ids : ['2633619'] with mtr_portfolio_ids : ['21019805'] and first list_photo_ids : [] new path : /proc/906067/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, brightness, blur_detection, rle_unique_nms_with_priority, crop_condition, thcl, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , BFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 2 ; length of list_pids : 2 ; length of list_args : 2 time to download the photos : 0.27119874954223633 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Tue Mar 4 18:20:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3177 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-03-04 18:20:32.946334: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-03-04 18:20:32.971330: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-03-04 18:20:32.973256: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fa994000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-03-04 18:20:32.973289: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-03-04 18:20:32.977444: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-03-04 18:20:33.257998: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x9c35a20 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-03-04 18:20:33.258054: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-03-04 18:20:33.258644: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-04 18:20:33.259317: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-04 18:20:33.263360: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-04 18:20:33.266191: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-04 18:20:33.266681: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-04 18:20:33.269910: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-04 18:20:33.271067: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-04 18:20:33.276035: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-04 18:20:33.277114: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-04 18:20:33.277199: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-04 18:20:33.277761: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-04 18:20:33.277777: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-04 18:20:33.277787: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-04 18:20:33.278698: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2725 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-03-04 18:20:33.602136: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-04 18:20:33.602267: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-04 18:20:33.602288: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-04 18:20:33.602308: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-04 18:20:33.602326: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-04 18:20:33.602344: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-04 18:20:33.602361: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-04 18:20:33.602380: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-04 18:20:33.603399: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-04 18:20:33.604724: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-04 18:20:33.604778: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-04 18:20:33.604797: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-04 18:20:33.604814: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-04 18:20:33.604832: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-04 18:20:33.604848: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-04 18:20:33.604864: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-04 18:20:33.604882: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-04 18:20:33.605882: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-04 18:20:33.605941: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-04 18:20:33.605953: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-04 18:20:33.605963: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-04 18:20:33.607685: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2725 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-03-04 18:20:41.129370: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-04 18:20:41.316061: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-04 18:20:42.725825: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.725894: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-04 18:20:42.726480: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.726497: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-04 18:20:42.733611: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.733633: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-04 18:20:42.734218: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.734234: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-04 18:20:42.740791: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.740812: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 466.56MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-04 18:20:42.741397: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.741413: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 466.56MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-04 18:20:42.773088: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.773159: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-04 18:20:42.773902: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.773922: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-04 18:20:42.780559: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.780609: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 243.25MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-04 18:20:42.781585: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.781613: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 243.25MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-04 18:20:42.816506: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.817106: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.819005: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.819599: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.863683: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.864293: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.866563: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.867177: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.896085: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.896696: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.898470: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.899686: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.906990: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.908101: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.909795: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.910376: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.915979: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.916523: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.918019: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.918559: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.944809: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.945365: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.945905: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.946445: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.949913: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.950459: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.965786: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.966338: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.966883: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.967485: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.979851: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.980444: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.981023: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.981605: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.985931: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.986480: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.991101: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:42.991690: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.003826: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.004418: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.008550: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.009138: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.009716: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.010299: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.011066: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.011655: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.022113: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.022691: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.023252: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.023793: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.024412: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.024997: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.040092: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.040678: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.069722: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.069775: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-03-04 18:20:43.070519: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.071397: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.078685: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.079513: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.088387: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.089777: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.105851: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.106479: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.107088: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.107675: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.111772: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.112365: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.112938: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.113479: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.114505: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.124578: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.125128: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.134062: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.134613: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.135174: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.135719: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.136271: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-04 18:20:43.136815: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.16G (2322464768 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 2 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -121.30156 max: 150.99453 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 7.963180541992188e-05 nb_pixel_total : 431 time to create 1 rle with old method : 0.0008180141448974609 length of segment : 33 time for calcul the mask position with numpy : 4.7206878662109375e-05 nb_pixel_total : 113 time to create 1 rle with old method : 0.00026726722717285156 length of segment : 26 time for calcul the mask position with numpy : 6.127357482910156e-05 nb_pixel_total : 946 time to create 1 rle with old method : 0.0017502307891845703 length of segment : 52 time for calcul the mask position with numpy : 5.9604644775390625e-05 nb_pixel_total : 993 time to create 1 rle with old method : 0.0018193721771240234 length of segment : 51 time for calcul the mask position with numpy : 5.7697296142578125e-05 nb_pixel_total : 964 time to create 1 rle with old method : 0.0017445087432861328 length of segment : 39 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.07109 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 0.00012302398681640625 nb_pixel_total : 2728 time to create 1 rle with old method : 0.004945278167724609 length of segment : 51 time for calcul the mask position with numpy : 0.0003292560577392578 nb_pixel_total : 292 time to create 1 rle with old method : 0.0005991458892822266 length of segment : 40 time for calcul the mask position with numpy : 7.319450378417969e-05 nb_pixel_total : 991 time to create 1 rle with old method : 0.0019249916076660156 length of segment : 33 time for calcul the mask position with numpy : 7.104873657226562e-05 nb_pixel_total : 739 time to create 1 rle with old method : 0.0014355182647705078 length of segment : 37 time for calcul the mask position with numpy : 6.985664367675781e-05 nb_pixel_total : 2383 time to create 1 rle with old method : 0.0031290054321289062 length of segment : 35 Processing 1 images image shape: (400, 400, 3) min: 5.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -107.56719 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 8.678436279296875e-05 nb_pixel_total : 3150 time to create 1 rle with old method : 0.003896474838256836 length of segment : 58 time for calcul the mask position with numpy : 6.747245788574219e-05 nb_pixel_total : 1152 time to create 1 rle with old method : 0.0016124248504638672 length of segment : 26 time for calcul the mask position with numpy : 9.322166442871094e-05 nb_pixel_total : 3702 time to create 1 rle with old method : 0.004755258560180664 length of segment : 51 time for calcul the mask position with numpy : 0.00010919570922851562 nb_pixel_total : 4699 time to create 1 rle with old method : 0.005894899368286133 length of segment : 51 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -120.39922 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 253.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 118.94766 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 5.316734313964844e-05 nb_pixel_total : 505 time to create 1 rle with old method : 0.0010745525360107422 length of segment : 17 time for calcul the mask position with numpy : 7.557868957519531e-05 nb_pixel_total : 1502 time to create 1 rle with old method : 0.002760171890258789 length of segment : 123 time for calcul the mask position with numpy : 0.0006613731384277344 nb_pixel_total : 66390 time to create 1 rle with old method : 0.07645869255065918 length of segment : 248 Processing 1 images image shape: (400, 400, 3) min: 27.00000 max: 189.00000 molded_images shape: (1, 640, 640, 3) min: -95.51641 max: 73.10781 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0012979507446289062 nb_pixel_total : 153431 time to create 1 rle with new method : 0.002056598663330078 length of segment : 401 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.26641 max: 151.03750 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 6.151199340820312e-05 nb_pixel_total : 1197 time to create 1 rle with old method : 0.0015649795532226562 length of segment : 39 time for calcul the mask position with numpy : 9.250640869140625e-05 nb_pixel_total : 2234 time to create 1 rle with old method : 0.0029790401458740234 length of segment : 48 time for calcul the mask position with numpy : 5.7697296142578125e-05 nb_pixel_total : 1054 time to create 1 rle with old method : 0.001306295394897461 length of segment : 50 time for calcul the mask position with numpy : 0.00019240379333496094 nb_pixel_total : 10336 time to create 1 rle with old method : 0.012254476547241211 length of segment : 132 time for calcul the mask position with numpy : 7.724761962890625e-05 nb_pixel_total : 1470 time to create 1 rle with old method : 0.001878499984741211 length of segment : 44 time for calcul the mask position with numpy : 7.462501525878906e-05 nb_pixel_total : 434 time to create 1 rle with old method : 0.0008988380432128906 length of segment : 22 time for calcul the mask position with numpy : 0.00010371208190917969 nb_pixel_total : 1236 time to create 1 rle with old method : 0.002450227737426758 length of segment : 40 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -114.58672 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 11 time for calcul the mask position with numpy : 0.00019288063049316406 nb_pixel_total : 9859 time to create 1 rle with old method : 0.01181173324584961 length of segment : 178 time for calcul the mask position with numpy : 6.532669067382812e-05 nb_pixel_total : 107 time to create 1 rle with old method : 0.00020265579223632812 length of segment : 9 time for calcul the mask position with numpy : 0.0002694129943847656 nb_pixel_total : 6198 time to create 1 rle with old method : 0.008169412612915039 length of segment : 244 time for calcul the mask position with numpy : 4.9114227294921875e-05 nb_pixel_total : 140 time to create 1 rle with old method : 0.00027489662170410156 length of segment : 15 time for calcul the mask position with numpy : 5.626678466796875e-05 nb_pixel_total : 725 time to create 1 rle with old method : 0.0011403560638427734 length of segment : 32 time for calcul the mask position with numpy : 0.00016164779663085938 nb_pixel_total : 6260 time to create 1 rle with old method : 0.007715463638305664 length of segment : 79 time for calcul the mask position with numpy : 4.649162292480469e-05 nb_pixel_total : 160 time to create 1 rle with old method : 0.0002799034118652344 length of segment : 11 time for calcul the mask position with numpy : 5.078315734863281e-05 nb_pixel_total : 860 time to create 1 rle with old method : 0.0012111663818359375 length of segment : 36 time for calcul the mask position with numpy : 4.315376281738281e-05 nb_pixel_total : 528 time to create 1 rle with old method : 0.0007109642028808594 length of segment : 25 time for calcul the mask position with numpy : 4.76837158203125e-05 nb_pixel_total : 636 time to create 1 rle with old method : 0.0009198188781738281 length of segment : 31 time for calcul the mask position with numpy : 3.4332275390625e-05 nb_pixel_total : 354 time to create 1 rle with old method : 0.000545501708984375 length of segment : 14 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -116.96563 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 9.1552734375e-05 nb_pixel_total : 1511 time to create 1 rle with old method : 0.0018968582153320312 length of segment : 102 time for calcul the mask position with numpy : 6.008148193359375e-05 nb_pixel_total : 1239 time to create 1 rle with old method : 0.001569509506225586 length of segment : 38 time for calcul the mask position with numpy : 4.172325134277344e-05 nb_pixel_total : 241 time to create 1 rle with old method : 0.0003542900085449219 length of segment : 20 time for calcul the mask position with numpy : 5.0067901611328125e-05 nb_pixel_total : 738 time to create 1 rle with old method : 0.0009725093841552734 length of segment : 37 time for calcul the mask position with numpy : 7.724761962890625e-05 nb_pixel_total : 1247 time to create 1 rle with old method : 0.0017511844635009766 length of segment : 43 time for calcul the mask position with numpy : 6.961822509765625e-05 nb_pixel_total : 1137 time to create 1 rle with old method : 0.0016291141510009766 length of segment : 41 time for calcul the mask position with numpy : 0.00025177001953125 nb_pixel_total : 10329 time to create 1 rle with old method : 0.01595473289489746 length of segment : 247 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 215.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 90.13516 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 5.364418029785156e-05 nb_pixel_total : 784 time to create 1 rle with old method : 0.0010266304016113281 length of segment : 44 time for calcul the mask position with numpy : 8.320808410644531e-05 nb_pixel_total : 631 time to create 1 rle with old method : 0.0009031295776367188 length of segment : 72 time for calcul the mask position with numpy : 0.00018215179443359375 nb_pixel_total : 3305 time to create 1 rle with old method : 0.004693031311035156 length of segment : 181 time for calcul the mask position with numpy : 9.751319885253906e-05 nb_pixel_total : 1462 time to create 1 rle with old method : 0.001829385757446289 length of segment : 76 time for calcul the mask position with numpy : 7.295608520507812e-05 nb_pixel_total : 737 time to create 1 rle with old method : 0.0010676383972167969 length of segment : 61 time for calcul the mask position with numpy : 4.363059997558594e-05 nb_pixel_total : 183 time to create 1 rle with old method : 0.00027441978454589844 length of segment : 22 time for calcul the mask position with numpy : 6.4849853515625e-05 nb_pixel_total : 761 time to create 1 rle with old method : 0.001079559326171875 length of segment : 62 time for calcul the mask position with numpy : 5.364418029785156e-05 nb_pixel_total : 476 time to create 1 rle with old method : 0.0006656646728515625 length of segment : 26 Processing 1 images image shape: (280, 400, 3) min: 26.00000 max: 192.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 78.24062 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0012569427490234375 nb_pixel_total : 104137 time to create 1 rle with old method : 0.13860273361206055 length of segment : 281 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 146.31094 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 6.031990051269531e-05 nb_pixel_total : 172 time to create 1 rle with old method : 0.0003910064697265625 length of segment : 17 time for calcul the mask position with numpy : 4.673004150390625e-05 nb_pixel_total : 275 time to create 1 rle with old method : 0.0006062984466552734 length of segment : 14 time for calcul the mask position with numpy : 6.318092346191406e-05 nb_pixel_total : 927 time to create 1 rle with old method : 0.0018956661224365234 length of segment : 41 time for calcul the mask position with numpy : 0.0001049041748046875 nb_pixel_total : 3135 time to create 1 rle with old method : 0.005824565887451172 length of segment : 49 time for calcul the mask position with numpy : 0.0005590915679931641 nb_pixel_total : 43201 time to create 1 rle with old method : 0.0489659309387207 length of segment : 243 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.47500 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 11 time for calcul the mask position with numpy : 3.9577484130859375e-05 nb_pixel_total : 230 time to create 1 rle with old method : 0.0003619194030761719 length of segment : 21 time for calcul the mask position with numpy : 3.24249267578125e-05 nb_pixel_total : 127 time to create 1 rle with old method : 0.0002090930938720703 length of segment : 17 time for calcul the mask position with numpy : 5.030632019042969e-05 nb_pixel_total : 479 time to create 1 rle with old method : 0.0006968975067138672 length of segment : 37 time for calcul the mask position with numpy : 3.743171691894531e-05 nb_pixel_total : 316 time to create 1 rle with old method : 0.00046062469482421875 length of segment : 26 time for calcul the mask position with numpy : 3.62396240234375e-05 nb_pixel_total : 276 time to create 1 rle with old method : 0.0004107952117919922 length of segment : 24 time for calcul the mask position with numpy : 3.981590270996094e-05 nb_pixel_total : 431 time to create 1 rle with old method : 0.0005960464477539062 length of segment : 34 time for calcul the mask position with numpy : 3.814697265625e-05 nb_pixel_total : 233 time to create 1 rle with old method : 0.00033164024353027344 length of segment : 25 time for calcul the mask position with numpy : 3.0994415283203125e-05 nb_pixel_total : 87 time to create 1 rle with old method : 0.00014328956604003906 length of segment : 11 time for calcul the mask position with numpy : 3.147125244140625e-05 nb_pixel_total : 119 time to create 1 rle with old method : 0.00021028518676757812 length of segment : 11 time for calcul the mask position with numpy : 3.62396240234375e-05 nb_pixel_total : 243 time to create 1 rle with old method : 0.0003638267517089844 length of segment : 22 time for calcul the mask position with numpy : 3.933906555175781e-05 nb_pixel_total : 311 time to create 1 rle with old method : 0.0005221366882324219 length of segment : 13 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.08047 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 3.8623809814453125e-05 nb_pixel_total : 258 time to create 1 rle with old method : 0.0003943443298339844 length of segment : 24 time for calcul the mask position with numpy : 3.552436828613281e-05 nb_pixel_total : 343 time to create 1 rle with old method : 0.0004715919494628906 length of segment : 24 time for calcul the mask position with numpy : 4.1484832763671875e-05 nb_pixel_total : 710 time to create 1 rle with old method : 0.0009071826934814453 length of segment : 42 time for calcul the mask position with numpy : 3.743171691894531e-05 nb_pixel_total : 418 time to create 1 rle with old method : 0.0005545616149902344 length of segment : 35 time for calcul the mask position with numpy : 5.125999450683594e-05 nb_pixel_total : 792 time to create 1 rle with old method : 0.0011529922485351562 length of segment : 56 time for calcul the mask position with numpy : 3.719329833984375e-05 nb_pixel_total : 308 time to create 1 rle with old method : 0.000423431396484375 length of segment : 36 time for calcul the mask position with numpy : 3.4809112548828125e-05 nb_pixel_total : 282 time to create 1 rle with old method : 0.0004153251647949219 length of segment : 27 time for calcul the mask position with numpy : 0.00011467933654785156 nb_pixel_total : 175 time to create 1 rle with old method : 0.00042247772216796875 length of segment : 29 time for calcul the mask position with numpy : 6.67572021484375e-05 nb_pixel_total : 999 time to create 1 rle with old method : 0.0013937950134277344 length of segment : 50 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 141.35000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.00040459632873535156 nb_pixel_total : 35103 time to create 1 rle with old method : 0.04305005073547363 length of segment : 221 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -118.71172 max: 150.85391 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 4.076957702636719e-05 nb_pixel_total : 416 time to create 1 rle with old method : 0.0005629062652587891 length of segment : 34 time for calcul the mask position with numpy : 3.814697265625e-05 nb_pixel_total : 83 time to create 1 rle with old method : 0.000156402587890625 length of segment : 23 time for calcul the mask position with numpy : 4.7206878662109375e-05 nb_pixel_total : 893 time to create 1 rle with old method : 0.0012085437774658203 length of segment : 59 time for calcul the mask position with numpy : 5.173683166503906e-05 nb_pixel_total : 896 time to create 1 rle with old method : 0.001207590103149414 length of segment : 55 time for calcul the mask position with numpy : 5.459785461425781e-05 nb_pixel_total : 929 time to create 1 rle with old method : 0.0012612342834472656 length of segment : 70 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.15313 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 5.030632019042969e-05 nb_pixel_total : 1121 time to create 1 rle with old method : 0.0014448165893554688 length of segment : 37 time for calcul the mask position with numpy : 6.413459777832031e-05 nb_pixel_total : 2699 time to create 1 rle with old method : 0.0033881664276123047 length of segment : 52 time for calcul the mask position with numpy : 6.842613220214844e-05 nb_pixel_total : 2143 time to create 1 rle with old method : 0.0028192996978759766 length of segment : 31 time for calcul the mask position with numpy : 0.00023651123046875 nb_pixel_total : 499 time to create 1 rle with old method : 0.0006418228149414062 length of segment : 55 time for calcul the mask position with numpy : 5.555152893066406e-05 nb_pixel_total : 628 time to create 1 rle with old method : 0.0008554458618164062 length of segment : 34 time for calcul the mask position with numpy : 0.00010418891906738281 nb_pixel_total : 2482 time to create 1 rle with old method : 0.0036423206329345703 length of segment : 93 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -113.41484 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 8.320808410644531e-05 nb_pixel_total : 1025 time to create 1 rle with old method : 0.0022079944610595703 length of segment : 25 time for calcul the mask position with numpy : 0.00014710426330566406 nb_pixel_total : 3141 time to create 1 rle with old method : 0.0057642459869384766 length of segment : 59 time for calcul the mask position with numpy : 0.0001533031463623047 nb_pixel_total : 4697 time to create 1 rle with old method : 0.008903980255126953 length of segment : 48 time for calcul the mask position with numpy : 8.893013000488281e-05 nb_pixel_total : 1333 time to create 1 rle with old method : 0.002690553665161133 length of segment : 45 time for calcul the mask position with numpy : 0.0001633167266845703 nb_pixel_total : 4496 time to create 1 rle with old method : 0.008897066116333008 length of segment : 54 time for calcul the mask position with numpy : 0.00018739700317382812 nb_pixel_total : 5426 time to create 1 rle with old method : 0.010356426239013672 length of segment : 49 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -121.21172 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 4.887580871582031e-05 nb_pixel_total : 235 time to create 1 rle with old method : 0.0004317760467529297 length of segment : 23 time for calcul the mask position with numpy : 5.221366882324219e-05 nb_pixel_total : 497 time to create 1 rle with old method : 0.0009284019470214844 length of segment : 25 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 251.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 114.10781 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 5.269050598144531e-05 nb_pixel_total : 535 time to create 1 rle with old method : 0.0009984970092773438 length of segment : 18 time for calcul the mask position with numpy : 7.390975952148438e-05 nb_pixel_total : 1070 time to create 1 rle with old method : 0.0017092227935791016 length of segment : 79 time for calcul the mask position with numpy : 7.200241088867188e-05 nb_pixel_total : 424 time to create 1 rle with old method : 0.0010499954223632812 length of segment : 24 time for calcul the mask position with numpy : 0.0007176399230957031 nb_pixel_total : 57644 time to create 1 rle with old method : 0.07112836837768555 length of segment : 257 Processing 1 images image shape: (400, 400, 3) min: 27.00000 max: 190.00000 molded_images shape: (1, 640, 640, 3) min: -89.42266 max: 69.60781 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -114.89141 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 5.173683166503906e-05 nb_pixel_total : 1188 time to create 1 rle with old method : 0.0015230178833007812 length of segment : 38 time for calcul the mask position with numpy : 6.961822509765625e-05 nb_pixel_total : 2350 time to create 1 rle with old method : 0.0030732154846191406 length of segment : 49 time for calcul the mask position with numpy : 3.790855407714844e-05 nb_pixel_total : 242 time to create 1 rle with old method : 0.0003368854522705078 length of segment : 22 time for calcul the mask position with numpy : 4.220008850097656e-05 nb_pixel_total : 994 time to create 1 rle with old method : 0.0012056827545166016 length of segment : 50 time for calcul the mask position with numpy : 3.552436828613281e-05 nb_pixel_total : 439 time to create 1 rle with old method : 0.0005960464477539062 length of segment : 24 time for calcul the mask position with numpy : 0.0001735687255859375 nb_pixel_total : 9805 time to create 1 rle with old method : 0.011711597442626953 length of segment : 134 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -115.00078 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 5.245208740234375e-05 nb_pixel_total : 118 time to create 1 rle with old method : 0.0002994537353515625 length of segment : 10 time for calcul the mask position with numpy : 0.0002238750457763672 nb_pixel_total : 8169 time to create 1 rle with old method : 0.014914751052856445 length of segment : 253 time for calcul the mask position with numpy : 0.0002295970916748047 nb_pixel_total : 3541 time to create 1 rle with old method : 0.005933523178100586 length of segment : 148 time for calcul the mask position with numpy : 0.0001533031463623047 nb_pixel_total : 5751 time to create 1 rle with old method : 0.007888078689575195 length of segment : 66 time for calcul the mask position with numpy : 5.1021575927734375e-05 nb_pixel_total : 472 time to create 1 rle with old method : 0.0006866455078125 length of segment : 17 time for calcul the mask position with numpy : 0.00017523765563964844 nb_pixel_total : 5526 time to create 1 rle with old method : 0.006947517395019531 length of segment : 68 time for calcul the mask position with numpy : 5.030632019042969e-05 nb_pixel_total : 728 time to create 1 rle with old method : 0.0009338855743408203 length of segment : 32 time for calcul the mask position with numpy : 3.8623809814453125e-05 nb_pixel_total : 153 time to create 1 rle with old method : 0.0002872943878173828 length of segment : 19 time for calcul the mask position with numpy : 6.151199340820312e-05 nb_pixel_total : 1340 time to create 1 rle with old method : 0.0017430782318115234 length of segment : 38 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -119.28203 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 8.320808410644531e-05 nb_pixel_total : 1457 time to create 1 rle with old method : 0.0018625259399414062 length of segment : 96 time for calcul the mask position with numpy : 5.841255187988281e-05 nb_pixel_total : 1128 time to create 1 rle with old method : 0.0020875930786132812 length of segment : 35 time for calcul the mask position with numpy : 7.939338684082031e-05 nb_pixel_total : 1243 time to create 1 rle with old method : 0.002505779266357422 length of segment : 44 time for calcul the mask position with numpy : 3.9577484130859375e-05 nb_pixel_total : 79 time to create 1 rle with old method : 0.00012874603271484375 length of segment : 19 time for calcul the mask position with numpy : 4.363059997558594e-05 nb_pixel_total : 753 time to create 1 rle with old method : 0.001210927963256836 length of segment : 37 time for calcul the mask position with numpy : 0.00025653839111328125 nb_pixel_total : 12692 time to create 1 rle with old method : 0.015409708023071289 length of segment : 238 time for calcul the mask position with numpy : 9.799003601074219e-05 nb_pixel_total : 2089 time to create 1 rle with old method : 0.0025568008422851562 length of segment : 54 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 211.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 89.06484 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 10 time for calcul the mask position with numpy : 6.890296936035156e-05 nb_pixel_total : 710 time to create 1 rle with old method : 0.0009322166442871094 length of segment : 78 time for calcul the mask position with numpy : 4.2438507080078125e-05 nb_pixel_total : 752 time to create 1 rle with old method : 0.0009593963623046875 length of segment : 40 time for calcul the mask position with numpy : 6.127357482910156e-05 nb_pixel_total : 835 time to create 1 rle with old method : 0.0011763572692871094 length of segment : 69 time for calcul the mask position with numpy : 4.220008850097656e-05 nb_pixel_total : 475 time to create 1 rle with old method : 0.0006670951843261719 length of segment : 26 time for calcul the mask position with numpy : 7.2479248046875e-05 nb_pixel_total : 1581 time to create 1 rle with old method : 0.0020797252655029297 length of segment : 76 time for calcul the mask position with numpy : 3.8623809814453125e-05 nb_pixel_total : 196 time to create 1 rle with old method : 0.0002772808074951172 length of segment : 23 time for calcul the mask position with numpy : 0.0001766681671142578 nb_pixel_total : 3608 time to create 1 rle with old method : 0.004476785659790039 length of segment : 189 time for calcul the mask position with numpy : 0.00010657310485839844 nb_pixel_total : 648 time to create 1 rle with old method : 0.001313924789428711 length of segment : 64 time for calcul the mask position with numpy : 8.487701416015625e-05 nb_pixel_total : 669 time to create 1 rle with old method : 0.0013217926025390625 length of segment : 49 time for calcul the mask position with numpy : 0.00021004676818847656 nb_pixel_total : 4338 time to create 1 rle with old method : 0.0056226253509521484 length of segment : 222 Processing 1 images image shape: (280, 400, 3) min: 22.00000 max: 190.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 79.26016 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0009517669677734375 nb_pixel_total : 103967 time to create 1 rle with old method : 0.12032365798950195 length of segment : 287 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 146.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 7.104873657226562e-05 nb_pixel_total : 3142 time to create 1 rle with old method : 0.0037910938262939453 length of segment : 48 time for calcul the mask position with numpy : 3.790855407714844e-05 nb_pixel_total : 169 time to create 1 rle with old method : 0.0002522468566894531 length of segment : 16 time for calcul the mask position with numpy : 3.2901763916015625e-05 nb_pixel_total : 113 time to create 1 rle with old method : 0.00020551681518554688 length of segment : 12 time for calcul the mask position with numpy : 3.5762786865234375e-05 nb_pixel_total : 298 time to create 1 rle with old method : 0.0004572868347167969 length of segment : 15 time for calcul the mask position with numpy : 4.601478576660156e-05 nb_pixel_total : 877 time to create 1 rle with old method : 0.001163482666015625 length of segment : 43 time for calcul the mask position with numpy : 0.00019288063049316406 nb_pixel_total : 5224 time to create 1 rle with old method : 0.006544589996337891 length of segment : 80 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.09219 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 15 time for calcul the mask position with numpy : 4.57763671875e-05 nb_pixel_total : 216 time to create 1 rle with old method : 0.00030922889709472656 length of segment : 21 time for calcul the mask position with numpy : 3.314018249511719e-05 nb_pixel_total : 77 time to create 1 rle with old method : 0.00013947486877441406 length of segment : 9 time for calcul the mask position with numpy : 3.123283386230469e-05 nb_pixel_total : 118 time to create 1 rle with old method : 0.00018715858459472656 length of segment : 17 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 478 time to create 1 rle with old method : 0.0006856918334960938 length of segment : 36 time for calcul the mask position with numpy : 3.62396240234375e-05 nb_pixel_total : 270 time to create 1 rle with old method : 0.0003998279571533203 length of segment : 26 time for calcul the mask position with numpy : 3.910064697265625e-05 nb_pixel_total : 477 time to create 1 rle with old method : 0.0006399154663085938 length of segment : 30 time for calcul the mask position with numpy : 4.0531158447265625e-05 nb_pixel_total : 381 time to create 1 rle with old method : 0.0006268024444580078 length of segment : 20 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 307 time to create 1 rle with old method : 0.0004096031188964844 length of segment : 36 time for calcul the mask position with numpy : 5.2928924560546875e-05 nb_pixel_total : 753 time to create 1 rle with old method : 0.0010190010070800781 length of segment : 37 time for calcul the mask position with numpy : 5.269050598144531e-05 nb_pixel_total : 95 time to create 1 rle with old method : 0.00021839141845703125 length of segment : 17 time for calcul the mask position with numpy : 5.4836273193359375e-05 nb_pixel_total : 257 time to create 1 rle with old method : 0.0005321502685546875 length of segment : 36 time for calcul the mask position with numpy : 3.910064697265625e-05 nb_pixel_total : 236 time to create 1 rle with old method : 0.00032973289489746094 length of segment : 22 time for calcul the mask position with numpy : 2.9802322387695312e-05 nb_pixel_total : 44 time to create 1 rle with old method : 8.845329284667969e-05 length of segment : 7 time for calcul the mask position with numpy : 3.1948089599609375e-05 nb_pixel_total : 140 time to create 1 rle with old method : 0.00024700164794921875 length of segment : 9 time for calcul the mask position with numpy : 3.3855438232421875e-05 nb_pixel_total : 86 time to create 1 rle with old method : 0.0001513957977294922 length of segment : 11 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.04141 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 4.792213439941406e-05 nb_pixel_total : 434 time to create 1 rle with old method : 0.000606536865234375 length of segment : 29 time for calcul the mask position with numpy : 7.319450378417969e-05 nb_pixel_total : 668 time to create 1 rle with old method : 0.0008611679077148438 length of segment : 42 time for calcul the mask position with numpy : 4.267692565917969e-05 nb_pixel_total : 543 time to create 1 rle with old method : 0.0007257461547851562 length of segment : 38 time for calcul the mask position with numpy : 3.7670135498046875e-05 nb_pixel_total : 354 time to create 1 rle with old method : 0.0004754066467285156 length of segment : 28 time for calcul the mask position with numpy : 3.719329833984375e-05 nb_pixel_total : 248 time to create 1 rle with old method : 0.0003478527069091797 length of segment : 22 time for calcul the mask position with numpy : 5.14984130859375e-05 nb_pixel_total : 455 time to create 1 rle with old method : 0.0006620883941650391 length of segment : 38 time for calcul the mask position with numpy : 6.890296936035156e-05 nb_pixel_total : 598 time to create 1 rle with old method : 0.0008304119110107422 length of segment : 48 time for calcul the mask position with numpy : 5.173683166503906e-05 nb_pixel_total : 929 time to create 1 rle with old method : 0.0012645721435546875 length of segment : 57 time for calcul the mask position with numpy : 5.936622619628906e-05 nb_pixel_total : 975 time to create 1 rle with old method : 0.0013539791107177734 length of segment : 59 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 141.72500 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 4.76837158203125e-05 nb_pixel_total : 15 time to create 1 rle with old method : 6.866455078125e-05 length of segment : 3 Detection mask done ! Trying to reset tf kernel 906997 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1984 tf kernel not reseted sub process len(results) : 165 len(list_Values) 165 None max_time_sub_proc : 3600 parent process len(results) : 0 len(list_Values) 165 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 3177 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Catched exception ! Connect or reconnect ! Number saved : None batch 1 Loaded 165 chid ids of type : 4228 Number RLEs to save : 9698 save missing photos in datou_result : time spend for datou_step_exec : 23.126878023147583 time spend to save output : 0.8182468414306641 total time spend for step 1 : 23.945124864578247 step2:brightness Tue Mar 4 18:20:53 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1741108829_906067_1340835968_c995a712a0db3d9a7c83edb61c43be1d.jpg treat image : temp/1741108829_906067_1340835963_c81f42ad46adc410cda676a127094170.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 2 time used for this insertion : 0.011738300323486328 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 2 time used for this insertion : 0.011220693588256836 save missing photos in datou_result : time spend for datou_step_exec : 0.800407886505127 time spend to save output : 0.02759242057800293 total time spend for step 2 : 0.8280003070831299 step3:blur_detection Tue Mar 4 18:20:54 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1741108829_906067_1340835968_c995a712a0db3d9a7c83edb61c43be1d.jpg resize: (1080, 1920) 1340835968 -7.514861538348438 treat image : temp/1741108829_906067_1340835963_c81f42ad46adc410cda676a127094170.jpg resize: (1080, 1920) 1340835963 -7.517188073894044 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 2 time used for this insertion : 0.013585329055786133 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 2 time used for this insertion : 0.011966466903686523 save missing photos in datou_result : time spend for datou_step_exec : 2.099853754043579 time spend to save output : 0.030407428741455078 total time spend for step 3 : 2.130261182785034 step4:rle_unique_nms_with_priority Tue Mar 4 18:20:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 165 chid ids of type : 4228 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 78 nb_hashtags : 5 time to prepare the origin masks : 1.2697887420654297 time for calcul the mask position with numpy : 0.05747079849243164 nb_pixel_total : 1584753 time to create 1 rle with new method : 0.11452698707580566 time for calcul the mask position with numpy : 0.009634256362915039 nb_pixel_total : 66291 time to create 1 rle with old method : 0.07232809066772461 time for calcul the mask position with numpy : 0.008474349975585938 nb_pixel_total : 991 time to create 1 rle with old method : 0.0011487007141113281 time for calcul the mask position with numpy : 0.008157730102539062 nb_pixel_total : 946 time to create 1 rle with old method : 0.0010700225830078125 time for calcul the mask position with numpy : 0.008307933807373047 nb_pixel_total : 113 time to create 1 rle with old method : 0.00014591217041015625 time for calcul the mask position with numpy : 0.008381366729736328 nb_pixel_total : 431 time to create 1 rle with old method : 0.0004973411560058594 time for calcul the mask position with numpy : 0.008379220962524414 nb_pixel_total : 149 time to create 1 rle with old method : 0.00022077560424804688 time for calcul the mask position with numpy : 0.00844717025756836 nb_pixel_total : 3150 time to create 1 rle with old method : 0.0035295486450195312 time for calcul the mask position with numpy : 0.008438825607299805 nb_pixel_total : 2728 time to create 1 rle with old method : 0.003082275390625 time for calcul the mask position with numpy : 0.00832223892211914 nb_pixel_total : 50 time to create 1 rle with old method : 7.176399230957031e-05 time for calcul the mask position with numpy : 0.008378267288208008 nb_pixel_total : 1502 time to create 1 rle with old method : 0.0016875267028808594 time for calcul the mask position with numpy : 0.00828242301940918 nb_pixel_total : 2383 time to create 1 rle with old method : 0.0027043819427490234 time for calcul the mask position with numpy : 0.008289575576782227 nb_pixel_total : 739 time to create 1 rle with old method : 0.0008246898651123047 time for calcul the mask position with numpy : 0.008510828018188477 nb_pixel_total : 1152 time to create 1 rle with old method : 0.0017125606536865234 time for calcul the mask position with numpy : 0.008467674255371094 nb_pixel_total : 3702 time to create 1 rle with old method : 0.004208803176879883 time for calcul the mask position with numpy : 0.008556127548217773 nb_pixel_total : 1006 time to create 1 rle with old method : 0.0012700557708740234 time for calcul the mask position with numpy : 0.008896112442016602 nb_pixel_total : 292 time to create 1 rle with old method : 0.0004012584686279297 time for calcul the mask position with numpy : 0.008673429489135742 nb_pixel_total : 505 time to create 1 rle with old method : 0.0006120204925537109 time for calcul the mask position with numpy : 0.00964665412902832 nb_pixel_total : 153431 time to create 1 rle with new method : 0.16332292556762695 time for calcul the mask position with numpy : 0.009542465209960938 nb_pixel_total : 10329 time to create 1 rle with old method : 0.011742591857910156 time for calcul the mask position with numpy : 0.008764982223510742 nb_pixel_total : 725 time to create 1 rle with old method : 0.0008869171142578125 time for calcul the mask position with numpy : 0.008659839630126953 nb_pixel_total : 9859 time to create 1 rle with old method : 0.011272192001342773 time for calcul the mask position with numpy : 0.008712530136108398 nb_pixel_total : 102 time to create 1 rle with old method : 0.00045371055603027344 time for calcul the mask position with numpy : 0.008723020553588867 nb_pixel_total : 183 time to create 1 rle with old method : 0.0002593994140625 time for calcul the mask position with numpy : 0.010190248489379883 nb_pixel_total : 738 time to create 1 rle with old method : 0.000904083251953125 time for calcul the mask position with numpy : 0.008485794067382812 nb_pixel_total : 757 time to create 1 rle with old method : 0.0009007453918457031 time for calcul the mask position with numpy : 0.008690118789672852 nb_pixel_total : 140 time to create 1 rle with old method : 0.00019860267639160156 time for calcul the mask position with numpy : 0.008487939834594727 nb_pixel_total : 636 time to create 1 rle with old method : 0.0008196830749511719 time for calcul the mask position with numpy : 0.008596420288085938 nb_pixel_total : 253 time to create 1 rle with old method : 0.0004010200500488281 time for calcul the mask position with numpy : 0.008865118026733398 nb_pixel_total : 10336 time to create 1 rle with old method : 0.01151728630065918 time for calcul the mask position with numpy : 0.00858306884765625 nb_pixel_total : 1239 time to create 1 rle with old method : 0.0016214847564697266 time for calcul the mask position with numpy : 0.009103536605834961 nb_pixel_total : 6036 time to create 1 rle with old method : 0.006907224655151367 time for calcul the mask position with numpy : 0.00874948501586914 nb_pixel_total : 158 time to create 1 rle with old method : 0.0002789497375488281 time for calcul the mask position with numpy : 0.008405208587646484 nb_pixel_total : 784 time to create 1 rle with old method : 0.000926971435546875 time for calcul the mask position with numpy : 0.009632110595703125 nb_pixel_total : 241 time to create 1 rle with old method : 0.00034046173095703125 time for calcul the mask position with numpy : 0.008654117584228516 nb_pixel_total : 460 time to create 1 rle with old method : 0.0005402565002441406 time for calcul the mask position with numpy : 0.00860905647277832 nb_pixel_total : 631 time to create 1 rle with old method : 0.0007431507110595703 time for calcul the mask position with numpy : 0.008984565734863281 nb_pixel_total : 3305 time to create 1 rle with old method : 0.0038805007934570312 time for calcul the mask position with numpy : 0.00870513916015625 nb_pixel_total : 93 time to create 1 rle with old method : 0.00016951560974121094 time for calcul the mask position with numpy : 0.008582353591918945 nb_pixel_total : 737 time to create 1 rle with old method : 0.0008764266967773438 time for calcul the mask position with numpy : 0.008638381958007812 nb_pixel_total : 292 time to create 1 rle with old method : 0.00035309791564941406 time for calcul the mask position with numpy : 0.008621692657470703 nb_pixel_total : 107 time to create 1 rle with old method : 0.0001456737518310547 time for calcul the mask position with numpy : 0.008649110794067383 nb_pixel_total : 28 time to create 1 rle with old method : 4.57763671875e-05 time for calcul the mask position with numpy : 0.008610248565673828 nb_pixel_total : 1511 time to create 1 rle with old method : 0.0017423629760742188 time for calcul the mask position with numpy : 0.008591890335083008 nb_pixel_total : 1197 time to create 1 rle with old method : 0.001402139663696289 time for calcul the mask position with numpy : 0.00860142707824707 nb_pixel_total : 1465 time to create 1 rle with old method : 0.0016813278198242188 time for calcul the mask position with numpy : 0.008587360382080078 nb_pixel_total : 1236 time to create 1 rle with old method : 0.0014138221740722656 time for calcul the mask position with numpy : 0.008563995361328125 nb_pixel_total : 2234 time to create 1 rle with old method : 0.002509593963623047 time for calcul the mask position with numpy : 0.008589744567871094 nb_pixel_total : 1247 time to create 1 rle with old method : 0.0014448165893554688 time for calcul the mask position with numpy : 0.008648872375488281 nb_pixel_total : 16 time to create 1 rle with old method : 7.605552673339844e-05 time for calcul the mask position with numpy : 0.00862264633178711 nb_pixel_total : 1054 time to create 1 rle with old method : 0.004495382308959961 time for calcul the mask position with numpy : 0.010243654251098633 nb_pixel_total : 1462 time to create 1 rle with old method : 0.001688241958618164 time for calcul the mask position with numpy : 0.008941411972045898 nb_pixel_total : 104137 time to create 1 rle with old method : 0.11560535430908203 time for calcul the mask position with numpy : 0.00852656364440918 nb_pixel_total : 39797 time to create 1 rle with old method : 0.044174909591674805 time for calcul the mask position with numpy : 0.00841522216796875 nb_pixel_total : 119 time to create 1 rle with old method : 0.00015234947204589844 time for calcul the mask position with numpy : 0.008400201797485352 nb_pixel_total : 35103 time to create 1 rle with old method : 0.041667938232421875 time for calcul the mask position with numpy : 0.008408784866333008 nb_pixel_total : 276 time to create 1 rle with old method : 0.0003287792205810547 time for calcul the mask position with numpy : 0.00837397575378418 nb_pixel_total : 316 time to create 1 rle with old method : 0.0003693103790283203 time for calcul the mask position with numpy : 0.008375883102416992 nb_pixel_total : 3135 time to create 1 rle with old method : 0.0035581588745117188 time for calcul the mask position with numpy : 0.008377552032470703 nb_pixel_total : 243 time to create 1 rle with old method : 0.00027942657470703125 time for calcul the mask position with numpy : 0.008383035659790039 nb_pixel_total : 343 time to create 1 rle with old method : 0.00040435791015625 time for calcul the mask position with numpy : 0.008381366729736328 nb_pixel_total : 479 time to create 1 rle with old method : 0.0005793571472167969 time for calcul the mask position with numpy : 0.008369207382202148 nb_pixel_total : 22 time to create 1 rle with old method : 4.3392181396484375e-05 time for calcul the mask position with numpy : 0.008371114730834961 nb_pixel_total : 308 time to create 1 rle with old method : 0.0003666877746582031 time for calcul the mask position with numpy : 0.008738040924072266 nb_pixel_total : 927 time to create 1 rle with old method : 0.0015859603881835938 time for calcul the mask position with numpy : 0.00966334342956543 nb_pixel_total : 431 time to create 1 rle with old method : 0.0007371902465820312 time for calcul the mask position with numpy : 0.009625434875488281 nb_pixel_total : 418 time to create 1 rle with old method : 0.0007081031799316406 time for calcul the mask position with numpy : 0.009629487991333008 nb_pixel_total : 275 time to create 1 rle with old method : 0.0004966259002685547 time for calcul the mask position with numpy : 0.00968027114868164 nb_pixel_total : 173 time to create 1 rle with old method : 0.0003590583801269531 time for calcul the mask position with numpy : 0.009685754776000977 nb_pixel_total : 87 time to create 1 rle with old method : 0.00016927719116210938 time for calcul the mask position with numpy : 0.009725570678710938 nb_pixel_total : 258 time to create 1 rle with old method : 0.0004553794860839844 time for calcul the mask position with numpy : 0.009724140167236328 nb_pixel_total : 172 time to create 1 rle with old method : 0.00022840499877929688 time for calcul the mask position with numpy : 0.008613348007202148 nb_pixel_total : 311 time to create 1 rle with old method : 0.0003921985626220703 time for calcul the mask position with numpy : 0.008527040481567383 nb_pixel_total : 710 time to create 1 rle with old method : 0.00083160400390625 time for calcul the mask position with numpy : 0.008694648742675781 nb_pixel_total : 792 time to create 1 rle with old method : 0.0009949207305908203 time for calcul the mask position with numpy : 0.008593082427978516 nb_pixel_total : 230 time to create 1 rle with old method : 0.00030159950256347656 time for calcul the mask position with numpy : 0.008636713027954102 nb_pixel_total : 233 time to create 1 rle with old method : 0.0003139972686767578 time for calcul the mask position with numpy : 0.008632421493530273 nb_pixel_total : 273 time to create 1 rle with old method : 0.0004062652587890625 time for calcul the mask position with numpy : 0.008611202239990234 nb_pixel_total : 127 time to create 1 rle with old method : 0.00017714500427246094 create new chi : 1.4205703735351562 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.01893448829650879 batch 1 Loaded 79 chid ids of type : 4230 Number RLEs to save : 9898 TO DO : save crop sub photo not yet done ! save time : 0.6391949653625488 nb_obj : 87 nb_hashtags : 5 time to prepare the origin masks : 3.068603277206421 time for calcul the mask position with numpy : 0.3778698444366455 nb_pixel_total : 1808626 time to create 1 rle with new method : 0.17043495178222656 time for calcul the mask position with numpy : 0.01015329360961914 nb_pixel_total : 1121 time to create 1 rle with old method : 0.0012891292572021484 time for calcul the mask position with numpy : 0.009654521942138672 nb_pixel_total : 125 time to create 1 rle with old method : 0.00019168853759765625 time for calcul the mask position with numpy : 0.009792327880859375 nb_pixel_total : 893 time to create 1 rle with old method : 0.0010597705841064453 time for calcul the mask position with numpy : 0.010120868682861328 nb_pixel_total : 83 time to create 1 rle with old method : 0.00011539459228515625 time for calcul the mask position with numpy : 0.009807586669921875 nb_pixel_total : 53 time to create 1 rle with old method : 9.632110595703125e-05 time for calcul the mask position with numpy : 0.005926370620727539 nb_pixel_total : 416 time to create 1 rle with old method : 0.0005030632019042969 time for calcul the mask position with numpy : 0.006028175354003906 nb_pixel_total : 3141 time to create 1 rle with old method : 0.0034220218658447266 time for calcul the mask position with numpy : 0.0060994625091552734 nb_pixel_total : 2699 time to create 1 rle with old method : 0.003069639205932617 time for calcul the mask position with numpy : 0.006435871124267578 nb_pixel_total : 57220 time to create 1 rle with old method : 0.06315946578979492 time for calcul the mask position with numpy : 0.006155729293823242 nb_pixel_total : 628 time to create 1 rle with old method : 0.0007379055023193359 time for calcul the mask position with numpy : 0.006199836730957031 nb_pixel_total : 1333 time to create 1 rle with old method : 0.0015797615051269531 time for calcul the mask position with numpy : 0.00611567497253418 nb_pixel_total : 1070 time to create 1 rle with old method : 0.0012776851654052734 time for calcul the mask position with numpy : 0.00623321533203125 nb_pixel_total : 2143 time to create 1 rle with old method : 0.0024993419647216797 time for calcul the mask position with numpy : 0.00635218620300293 nb_pixel_total : 235 time to create 1 rle with old method : 0.0003032684326171875 time for calcul the mask position with numpy : 0.006384372711181641 nb_pixel_total : 2482 time to create 1 rle with old method : 0.0028464794158935547 time for calcul the mask position with numpy : 0.0063076019287109375 nb_pixel_total : 222 time to create 1 rle with old method : 0.00037026405334472656 time for calcul the mask position with numpy : 0.007642030715942383 nb_pixel_total : 4697 time to create 1 rle with old method : 0.005346059799194336 time for calcul the mask position with numpy : 0.010024309158325195 nb_pixel_total : 1025 time to create 1 rle with old method : 0.0011854171752929688 time for calcul the mask position with numpy : 0.01001119613647461 nb_pixel_total : 820 time to create 1 rle with old method : 0.0009946823120117188 time for calcul the mask position with numpy : 0.011785745620727539 nb_pixel_total : 499 time to create 1 rle with old method : 0.0005826950073242188 time for calcul the mask position with numpy : 0.010226249694824219 nb_pixel_total : 424 time to create 1 rle with old method : 0.0005130767822265625 time for calcul the mask position with numpy : 0.009717941284179688 nb_pixel_total : 497 time to create 1 rle with old method : 0.0006036758422851562 time for calcul the mask position with numpy : 0.009607791900634766 nb_pixel_total : 535 time to create 1 rle with old method : 0.0006401538848876953 time for calcul the mask position with numpy : 0.01009678840637207 nb_pixel_total : 12692 time to create 1 rle with old method : 0.014339447021484375 time for calcul the mask position with numpy : 0.011204004287719727 nb_pixel_total : 728 time to create 1 rle with old method : 0.0008420944213867188 time for calcul the mask position with numpy : 0.01007986068725586 nb_pixel_total : 196 time to create 1 rle with old method : 0.0002415180206298828 time for calcul the mask position with numpy : 0.009363889694213867 nb_pixel_total : 8169 time to create 1 rle with old method : 0.009323358535766602 time for calcul the mask position with numpy : 0.010548114776611328 nb_pixel_total : 140 time to create 1 rle with old method : 0.00033736228942871094 time for calcul the mask position with numpy : 0.01027226448059082 nb_pixel_total : 753 time to create 1 rle with old method : 0.0008699893951416016 time for calcul the mask position with numpy : 0.01114797592163086 nb_pixel_total : 153 time to create 1 rle with old method : 0.00021982192993164062 time for calcul the mask position with numpy : 0.011054039001464844 nb_pixel_total : 472 time to create 1 rle with old method : 0.0005991458892822266 time for calcul the mask position with numpy : 0.011081933975219727 nb_pixel_total : 9504 time to create 1 rle with old method : 0.010718584060668945 time for calcul the mask position with numpy : 0.01044011116027832 nb_pixel_total : 1128 time to create 1 rle with old method : 0.0012958049774169922 time for calcul the mask position with numpy : 0.0068531036376953125 nb_pixel_total : 5514 time to create 1 rle with old method : 0.006325960159301758 time for calcul the mask position with numpy : 0.006748199462890625 nb_pixel_total : 125 time to create 1 rle with old method : 0.0002849102020263672 time for calcul the mask position with numpy : 0.006829023361206055 nb_pixel_total : 752 time to create 1 rle with old method : 0.0008938312530517578 time for calcul the mask position with numpy : 0.006459474563598633 nb_pixel_total : 2089 time to create 1 rle with old method : 0.0024073123931884766 time for calcul the mask position with numpy : 0.0065882205963134766 nb_pixel_total : 79 time to create 1 rle with old method : 0.00011420249938964844 time for calcul the mask position with numpy : 0.0065004825592041016 nb_pixel_total : 663 time to create 1 rle with old method : 0.0008413791656494141 time for calcul the mask position with numpy : 0.006590366363525391 nb_pixel_total : 1340 time to create 1 rle with old method : 0.0016009807586669922 time for calcul the mask position with numpy : 0.006571054458618164 nb_pixel_total : 890 time to create 1 rle with old method : 0.0011219978332519531 time for calcul the mask position with numpy : 0.006670951843261719 nb_pixel_total : 3608 time to create 1 rle with old method : 0.004180431365966797 time for calcul the mask position with numpy : 0.0064389705657958984 nb_pixel_total : 710 time to create 1 rle with old method : 0.0008451938629150391 time for calcul the mask position with numpy : 0.00641942024230957 nb_pixel_total : 835 time to create 1 rle with old method : 0.0010058879852294922 time for calcul the mask position with numpy : 0.006349802017211914 nb_pixel_total : 362 time to create 1 rle with old method : 0.0004467964172363281 time for calcul the mask position with numpy : 0.006488323211669922 nb_pixel_total : 71 time to create 1 rle with old method : 0.0001323223114013672 time for calcul the mask position with numpy : 0.006573677062988281 nb_pixel_total : 242 time to create 1 rle with old method : 0.0002980232238769531 time for calcul the mask position with numpy : 0.006834745407104492 nb_pixel_total : 439 time to create 1 rle with old method : 0.0005252361297607422 time for calcul the mask position with numpy : 0.006344795227050781 nb_pixel_total : 118 time to create 1 rle with old method : 0.0001633167266845703 time for calcul the mask position with numpy : 0.006112337112426758 nb_pixel_total : 1457 time to create 1 rle with old method : 0.001705169677734375 time for calcul the mask position with numpy : 0.00670313835144043 nb_pixel_total : 1188 time to create 1 rle with old method : 0.001416921615600586 time for calcul the mask position with numpy : 0.0064563751220703125 nb_pixel_total : 2349 time to create 1 rle with old method : 0.0028426647186279297 time for calcul the mask position with numpy : 0.006838083267211914 nb_pixel_total : 1243 time to create 1 rle with old method : 0.0015380382537841797 time for calcul the mask position with numpy : 0.006620883941650391 nb_pixel_total : 994 time to create 1 rle with old method : 0.0011339187622070312 time for calcul the mask position with numpy : 0.00633692741394043 nb_pixel_total : 1581 time to create 1 rle with old method : 0.0018734931945800781 time for calcul the mask position with numpy : 0.007024049758911133 nb_pixel_total : 103967 time to create 1 rle with old method : 0.11623167991638184 time for calcul the mask position with numpy : 0.0075457096099853516 nb_pixel_total : 113 time to create 1 rle with old method : 0.0001735687255859375 time for calcul the mask position with numpy : 0.007541656494140625 nb_pixel_total : 77 time to create 1 rle with old method : 0.00012373924255371094 time for calcul the mask position with numpy : 0.00758051872253418 nb_pixel_total : 86 time to create 1 rle with old method : 0.00014281272888183594 time for calcul the mask position with numpy : 0.007529020309448242 nb_pixel_total : 44 time to create 1 rle with old method : 8.869171142578125e-05 time for calcul the mask position with numpy : 0.00827789306640625 nb_pixel_total : 270 time to create 1 rle with old method : 0.0004291534423828125 time for calcul the mask position with numpy : 0.00858759880065918 nb_pixel_total : 3142 time to create 1 rle with old method : 0.0035970211029052734 time for calcul the mask position with numpy : 0.006818532943725586 nb_pixel_total : 354 time to create 1 rle with old method : 0.00044417381286621094 time for calcul the mask position with numpy : 0.007379055023193359 nb_pixel_total : 236 time to create 1 rle with old method : 0.00029206275939941406 time for calcul the mask position with numpy : 0.00632786750793457 nb_pixel_total : 478 time to create 1 rle with old method : 0.0005838871002197266 time for calcul the mask position with numpy : 0.006026506423950195 nb_pixel_total : 5224 time to create 1 rle with old method : 0.005829572677612305 time for calcul the mask position with numpy : 0.006259441375732422 nb_pixel_total : 257 time to create 1 rle with old method : 0.0003209114074707031 time for calcul the mask position with numpy : 0.0065648555755615234 nb_pixel_total : 95 time to create 1 rle with old method : 0.0001327991485595703 time for calcul the mask position with numpy : 0.006609439849853516 nb_pixel_total : 753 time to create 1 rle with old method : 0.0008842945098876953 time for calcul the mask position with numpy : 0.006559848785400391 nb_pixel_total : 434 time to create 1 rle with old method : 0.0005099773406982422 time for calcul the mask position with numpy : 0.006493806838989258 nb_pixel_total : 877 time to create 1 rle with old method : 0.0011196136474609375 time for calcul the mask position with numpy : 0.006669282913208008 nb_pixel_total : 15 time to create 1 rle with old method : 4.1484832763671875e-05 time for calcul the mask position with numpy : 0.0065228939056396484 nb_pixel_total : 477 time to create 1 rle with old method : 0.0005712509155273438 time for calcul the mask position with numpy : 0.006686687469482422 nb_pixel_total : 543 time to create 1 rle with old method : 0.0006513595581054688 time for calcul the mask position with numpy : 0.0067958831787109375 nb_pixel_total : 298 time to create 1 rle with old method : 0.00036406517028808594 time for calcul the mask position with numpy : 0.006499290466308594 nb_pixel_total : 598 time to create 1 rle with old method : 0.0007061958312988281 time for calcul the mask position with numpy : 0.006768465042114258 nb_pixel_total : 454 time to create 1 rle with old method : 0.0005559921264648438 time for calcul the mask position with numpy : 0.006665945053100586 nb_pixel_total : 381 time to create 1 rle with old method : 0.0004661083221435547 time for calcul the mask position with numpy : 0.008541345596313477 nb_pixel_total : 248 time to create 1 rle with old method : 0.0003139972686767578 time for calcul the mask position with numpy : 0.008550643920898438 nb_pixel_total : 169 time to create 1 rle with old method : 0.00021791458129882812 time for calcul the mask position with numpy : 0.008530139923095703 nb_pixel_total : 668 time to create 1 rle with old method : 0.0008182525634765625 time for calcul the mask position with numpy : 0.00846719741821289 nb_pixel_total : 535 time to create 1 rle with old method : 0.0006597042083740234 time for calcul the mask position with numpy : 0.008734941482543945 nb_pixel_total : 216 time to create 1 rle with old method : 0.0002689361572265625 time for calcul the mask position with numpy : 0.008501052856445312 nb_pixel_total : 307 time to create 1 rle with old method : 0.0003733634948730469 time for calcul the mask position with numpy : 0.008550167083740234 nb_pixel_total : 455 time to create 1 rle with old method : 0.0005707740783691406 time for calcul the mask position with numpy : 0.008528947830200195 nb_pixel_total : 118 time to create 1 rle with old method : 0.00016307830810546875 time for calcul the mask position with numpy : 0.011515617370605469 nb_pixel_total : 140 time to create 1 rle with old method : 0.0001773834228515625 create new chi : 1.5476422309875488 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.001528024673461914 batch 1 Loaded 88 chid ids of type : 4230 Number RLEs to save : 9460 TO DO : save crop sub photo not yet done ! save time : 0.6381561756134033 map_output_result : {1340835968: (0.0, 'Should be the crop_list due to order', 0.0), 1340835963: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1340835968, 1340835963] Looping around the photos to save general results len do output : 2 /1340835968.Didn't retrieve data . /1340835963.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2633619') ('4234', '21019805', '1340835968', None, None, None, None, None, '2633619') ('4234', None, None, None, None, None, None, None, '2633619') ('4234', '21019805', '1340835963', None, None, None, None, None, '2633619') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.013790369033813477 save_final save missing photos in datou_result : time spend for datou_step_exec : 8.997870922088623 time spend to save output : 0.014096498489379883 total time spend for step 4 : 9.011967420578003 step5:crop_condition Tue Mar 4 18:21:05 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4230 Loading chi in step crop for list_pids : 2 ! batch 1 Loaded 167 chid ids of type : 4230 begin to crop the class : papier param for this class : {'min_score': 0.6} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 55 About to insert : list_path_to_insert length 55 new photo from crops ! About to upload 55 photos upload in portfolio : 4869462 init cache_photo without model_param we have 55 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741108866_906067 we have uploaded 55 photos in the portfolio 4869462 time of upload the photos Elapsed time : 15.1239333152771 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.6} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 20 About to insert : list_path_to_insert length 20 new photo from crops ! About to upload 20 photos upload in portfolio : 4869462 init cache_photo without model_param we have 20 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741108882_906067 we have uploaded 20 photos in the portfolio 4869462 time of upload the photos Elapsed time : 4.266573190689087 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.6} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.6} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 10 About to insert : list_path_to_insert length 10 new photo from crops ! About to upload 10 photos upload in portfolio : 4869462 init cache_photo without model_param we have 10 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741108886_906067 we have uploaded 10 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.6415255069732666 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.6} filtre for class : autre hashtag_id of this class : 494826614 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 32 About to insert : list_path_to_insert length 32 new photo from crops ! About to upload 32 photos upload in portfolio : 4869462 init cache_photo without model_param we have 32 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741108890_906067 we have uploaded 32 photos in the portfolio 4869462 time of upload the photos Elapsed time : 7.277390956878662 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.6} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.6} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1340835968, 1340835963] Looping around the photos to save general results len do output : 117 /1340843652Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843653Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843654Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843655Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843656Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843657Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843658Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843659Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843660Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843661Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843662Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843663Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843664Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843665Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843666Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843667Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843668Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843669Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843670Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843671Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843672Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843673Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843674Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843676Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843677Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843678Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843679Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843680Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843681Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843682Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843683Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843684Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843685Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843686Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843687Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843688Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843689Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843690Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843691Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843692Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843693Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843694Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843695Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843696Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843697Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843698Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843699Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843700Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843701Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843702Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843703Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843704Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843705Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843706Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843707Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843710Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843711Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843712Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843713Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843714Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843715Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843716Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843717Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843718Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843719Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843720Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843721Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843722Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843723Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843724Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843725Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843726Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843727Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843728Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843729Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843731Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843732Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843733Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843734Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843735Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843736Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843737Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843738Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843739Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843741Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843746Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843747Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843748Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843749Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843750Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843751Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843752Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843753Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843754Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843755Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843756Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843757Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843758Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843759Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843760Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843761Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843762Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843763Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843764Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843765Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843766Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843767Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843768Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843769Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843772Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843773Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843774Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843775Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843776Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843777Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843778Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340843779Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2633619') ('4234', '21019805', '1340835968', None, None, None, None, None, '2633619') ('4234', None, None, None, None, None, None, None, '2633619') ('4234', '21019805', '1340835963', None, None, None, None, None, '2633619') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 353 time used for this insertion : 0.030515670776367188 save_final save missing photos in datou_result : time spend for datou_step_exec : 31.697308778762817 time spend to save output : 0.03499317169189453 total time spend for step 5 : 31.732301950454712 step6:thcl Tue Mar 4 18:21:37 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! we are using the classfication for only one thcl 3237 time to import caffe and check if the image exist : 0.008372783660888672 time to convert the images to numpy array : 0.03571367263793945 time to import caffe and check if the image exist : 0.009679555892944336 time to convert the images to numpy array : 0.034330129623413086 time to import caffe and check if the image exist : 0.015439748764038086 time to convert the images to numpy array : 0.02934741973876953 time to import caffe and check if the image exist : 0.011140108108520508 time to convert the images to numpy array : 0.03589653968811035 time to import caffe and check if the image exist : 0.01101994514465332 time to convert the images to numpy array : 0.03493618965148926 time to import caffe and check if the image exist : 0.012989282608032227 time to convert the images to numpy array : 0.03459978103637695 time to import caffe and check if the image exist : 0.007910490036010742 time to convert the images to numpy array : 0.0419774055480957 time to import caffe and check if the image exist : 0.016764163970947266 time to convert the images to numpy array : 0.03412151336669922 time to import caffe and check if the image exist : 0.010881423950195312 time to convert the images to numpy array : 0.04104280471801758 time to import caffe and check if the image exist : 0.020639657974243164 time to convert the images to numpy array : 0.03180861473083496 total time to convert the images to numpy array : 0.4064595699310303 list photo_ids error: [] list photo_ids correct : [1340843677, 1340843678, 1340843679, 1340843680, 1340843681, 1340843682, 1340843683, 1340843684, 1340843685, 1340843686, 1340843687, 1340843688, 1340843701, 1340843702, 1340843703, 1340843704, 1340843705, 1340843706, 1340843707, 1340843710, 1340843711, 1340843712, 1340843713, 1340843714, 1340843664, 1340843665, 1340843666, 1340843667, 1340843668, 1340843669, 1340843670, 1340843671, 1340843672, 1340843673, 1340843674, 1340843676, 1340843741, 1340843746, 1340843747, 1340843748, 1340843749, 1340843750, 1340843751, 1340843752, 1340843753, 1340843754, 1340843755, 1340843756, 1340843769, 1340843772, 1340843773, 1340843774, 1340843775, 1340843776, 1340843777, 1340843778, 1340843779, 1340843757, 1340843758, 1340843759, 1340843760, 1340843761, 1340843762, 1340843763, 1340843764, 1340843765, 1340843766, 1340843767, 1340843768, 1340843652, 1340843653, 1340843654, 1340843655, 1340843656, 1340843657, 1340843658, 1340843659, 1340843660, 1340843661, 1340843662, 1340843663, 1340843715, 1340843716, 1340843717, 1340843718, 1340843719, 1340843720, 1340843721, 1340843722, 1340843723, 1340843724, 1340843725, 1340843726, 1340843689, 1340843690, 1340843691, 1340843692, 1340843693, 1340843694, 1340843695, 1340843696, 1340843697, 1340843698, 1340843699, 1340843700, 1340843727, 1340843728, 1340843729, 1340843731, 1340843732, 1340843733, 1340843734, 1340843735, 1340843736, 1340843737, 1340843738, 1340843739] number of photos to traite : 117 try to delete the photos incorrect in DB tagging for thcl : 3237 To do loadFromThcl(), then load ParamDescType : thcl3237 thcls : [{'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'}] thcl {'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'} Update svm_hashtag_type_desc : 5561 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) To loadFromThcl() : net_5561 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3177 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_rubbia_refus_2500 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_rubbia_refus_2500 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_rubbia_refus_2500 /data/models_weight/learn_rubbia_refus_2500/caffemodel size_local : 94358479 size in s3 : 94358479 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:16 caffemodel already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt size_local : 32544 size in s3 : 32544 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:15 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:09:52 mean.npy already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/synset_words.txt size_local : 334 size in s3 : 334 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:10:06 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt caffemodel_filename : /data/models_weight/learn_rubbia_refus_2500/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 2958 max_wait_temp : 1 max_wait : 0 dict_keys(['prob', 'pool5']) time used to do the prepocess of the images : 0.5746212005615234 time used to do the prediction : 0.527881383895874 save descriptor for thcl : 3237 time to traite the descriptors : 0.6816706657409668 Catched exception ! Connect or reconnect ! storage_type for insertDescriptorsMulti : 3 To insert : 1340843677 To insert : 1340843678 To insert : 1340843679 To insert : 1340843680 To insert : 1340843681 To insert : 1340843682 To insert : 1340843683 To insert : 1340843684 To insert : 1340843685 To insert : 1340843686 To insert : 1340843687 To insert : 1340843688 To insert : 1340843701 To insert : 1340843702 To insert : 1340843703 To insert : 1340843704 To insert : 1340843705 To insert : 1340843706 To insert : 1340843707 To insert : 1340843710 To insert : 1340843711 To insert : 1340843712 To insert : 1340843713 To insert : 1340843714 To insert : 1340843664 To insert : 1340843665 To insert : 1340843666 To insert : 1340843667 To insert : 1340843668 To insert : 1340843669 To insert : 1340843670 To insert : 1340843671 To insert : 1340843672 To insert : 1340843673 To insert : 1340843674 To insert : 1340843676 To insert : 1340843741 To insert : 1340843746 To insert : 1340843747 To insert : 1340843748 To insert : 1340843749 To insert : 1340843750 To insert : 1340843751 To insert : 1340843752 To insert : 1340843753 To insert : 1340843754 To insert : 1340843755 To insert : 1340843756 To insert : 1340843769 To insert : 1340843772 To insert : 1340843773 To insert : 1340843774 To insert : 1340843775 To insert : 1340843776 To insert : 1340843777 To insert : 1340843778 To insert : 1340843779 To insert : 1340843757 To insert : 1340843758 To insert : 1340843759 To insert : 1340843760 To insert : 1340843761 To insert : 1340843762 To insert : 1340843763 To insert : 1340843764 To insert : 1340843765 To insert : 1340843766 To insert : 1340843767 To insert : 1340843768 To insert : 1340843652 To insert : 1340843653 To insert : 1340843654 To insert : 1340843655 To insert : 1340843656 To insert : 1340843657 To insert : 1340843658 To insert : 1340843659 To insert : 1340843660 To insert : 1340843661 To insert : 1340843662 To insert : 1340843663 To insert : 1340843715 To insert : 1340843716 To insert : 1340843717 To insert : 1340843718 To insert : 1340843719 To insert : 1340843720 To insert : 1340843721 To insert : 1340843722 To insert : 1340843723 To insert : 1340843724 To insert : 1340843725 To insert : 1340843726 To insert : 1340843689 To insert : 1340843690 To insert : 1340843691 To insert : 1340843692 To insert : 1340843693 To insert : 1340843694 To insert : 1340843695 To insert : 1340843696 To insert : 1340843697 To insert : 1340843698 To insert : 1340843699 To insert : 1340843700 To insert : 1340843727 To insert : 1340843728 To insert : 1340843729 To insert : 1340843731 To insert : 1340843732 To insert : 1340843733 To insert : 1340843734 To insert : 1340843735 To insert : 1340843736 To insert : 1340843737 To insert : 1340843738 To insert : 1340843739 time to insert the descriptors : 22.632664680480957 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos SAVE THCL : begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 182 time used for this insertion : 0.01870107650756836 save missing photos in datou_result : time spend for datou_step_exec : 29.094566583633423 time spend to save output : 0.0645301342010498 total time spend for step 6 : 29.159096717834473 step7:ventilate_hashtags_in_portfolio Tue Mar 4 18:22:06 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 21019805 get user id for portfolio 21019805 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21019805 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','Papier','autre','Tetrapak','metal','PEHD','Film_plastique','pet_fonce','carton','PET_fonce','environnement','refus','mal_croppe','pet_clair','Carton','flou','papier','PET_clair')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21019805 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','Papier','autre','Tetrapak','metal','PEHD','Film_plastique','pet_fonce','carton','PET_fonce','environnement','refus','mal_croppe','pet_clair','Carton','flou','papier','PET_clair')) AND mptpi.`min_score`=0.6 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21019805 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','Papier','autre','Tetrapak','metal','PEHD','Film_plastique','pet_fonce','carton','PET_fonce','environnement','refus','mal_croppe','pet_clair','Carton','flou','papier','PET_clair')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21019805 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','Papier','autre','Tetrapak','metal','PEHD','Film_plastique','pet_fonce','carton','PET_fonce','environnement','refus','mal_croppe','pet_clair','Carton','flou','papier','PET_clair')) AND mptpi.`min_score`=0.6 To do lien utilise dans velours : https://www.fotonower.com/velours/21020449,21020460,21020446,21020447,21020448,21020450,21020453,21020458,21020454,21020455,21020456,21020461,21020459?tags=pehd,papier,autre,tetrapak,metal,film_plastique,pet_fonce,carton,environnement,refus,mal_croppe,pet_clair,flou&datou_id_consolidate=4235&port_consolidate=21019805 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1340835968, 1340835963] Looping around the photos to save general results len do output : 1 /21019805. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2633619') ('4234', '21019805', '1340835968', None, None, None, None, None, '2633619') ('4234', None, None, None, None, None, None, None, '2633619') ('4234', '21019805', '1340835963', None, None, None, None, None, '2633619') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.012644529342651367 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.3048288822174072 time spend to save output : 0.012903213500976562 total time spend for step 7 : 2.317732095718384 step8:final Tue Mar 4 18:22:08 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1340835968: ('0.24126676634409014',), 1340835963: ('0.24126676634409014',)} new output for save of step final : {1340835968: ('0.24126676634409014',), 1340835963: ('0.24126676634409014',)} [1340835968, 1340835963] Looping around the photos to save general results len do output : 2 /1340835968.Didn't retrieve data . /1340835963.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2633619') ('4234', '21019805', '1340835968', None, None, None, None, None, '2633619') ('4234', None, None, None, None, None, None, None, '2633619') ('4234', '21019805', '1340835963', None, None, None, None, None, '2633619') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.011621952056884766 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.12511301040649414 time spend to save output : 0.01181793212890625 total time spend for step 8 : 0.1369309425354004 step9:velours_tree Tue Mar 4 18:22:08 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure list_portfolios : 21020449,21020460,21020446,21020447,21020448,21020450,21020453,21020458,21020454,21020455,21020456,21020461,21020459 photo desc type : 5561 - Retrieving photos to tag... query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21020449 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21020460 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21020446 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21020447 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21020448 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21020450 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21020453 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21020458 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21020454 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21020455 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21020456 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21020461 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21020459 ORDER BY ph.size desc - Loading descriptors... Size : 2048 len(descriptors) : 117 Compute structured hierarchical clustering... ward : AgglomerativeClustering(n_clusters=6) ward.labels_ : [1 1 1 0 4 1 1 0 2 0 0 0 0 2 0 4 0 0 5 4 4 1 1 0 0 2 1 1 0 4 1 1 0 1 2 0 2 0 0 0 2 0 4 0 0 5 4 2 4 1 1 2 0 0 0 3 3 0 0 0 0 0 3 1 2 3 0 0 0 0 0 0 1 2 0 1 4 4 1 1 0 4 4 4 1 2 3 2 1 4 1 4 4 4 2 2 2 2 2 2 2 2 3 3 2 4 1 4 5 1 4 4 2 2 3 2 2] Elapsed time: 0.02283954620361328 graph_id used : 81430 - Beta version, working pretty good on 11-5-16 ! https://www.fotonower.com/velours/21020449,21020460,21020446,21020447,21020448,21020450,21020453,21020458,21020454,21020455,21020456,21020461,21020459?tags=pehd,papier,autre,tetrapak,metal,film_plastique,pet_fonce,carton,environnement,refus,mal_croppe,pet_clair,flou&datou_id_consolidate=4235&port_consolidate=21019805&tree_id=81430 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : velours_tree we use saveGeneral [1340835968, 1340835963] Looping around the photos to save general results len do output : 1 /21019805Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2633619') ('4234', '21019805', '1340835968', None, None, None, None, None, '2633619') ('4234', None, None, None, None, None, None, None, '2633619') ('4234', '21019805', '1340835963', None, None, None, None, None, '2633619') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.08416414260864258 save_final save missing photos in datou_result : time spend for datou_step_exec : 8.537298679351807 time spend to save output : 0.08437633514404297 total time spend for step 9 : 8.62167501449585 step10:send_mail_cod Tue Mar 4 18:22:17 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Qualipapia_P21019805_04-03-2025_18_22_17.pdf 21020431 imagette210204311741108937 21020442 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette210204421741108937 21020428 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette210204281741108938 21020429 imagette210204291741108939 21020430 imagette210204301741108939 21020432 imagette210204321741108939 21020435 imagette210204351741108939 21020440 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette210204401741108939 21020437 imagette210204371741108940 21020438 imagette210204381741108940 21020443 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette210204431741108940 21020441 imagette210204411741108940 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=21019805 and hashtag_type = 4230 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/21020449,21020460,21020446,21020447,21020448,21020450,21020453,21020458,21020454,21020455,21020456,21020461,21020459?tags=pehd,papier,autre,tetrapak,metal,film_plastique,pet_fonce,carton,environnement,refus,mal_croppe,pet_clair,flou&datou_id_consolidate=4235&port_consolidate=21019805&tree_id=81430 your option no_mail is active, we will not send the real mail to your client args[1340835968] : ((1340835968, -7.514861538348438, 492609224), (1340835968, -0.40080927878736333, 496442774), '0.24126676634409014') We are sending mail with results at cod@fotonower.com args[1340835963] : ((1340835963, -7.517188073894044, 492609224), (1340835963, -0.3983289112179825, 496442774), '0.24126676634409014') We are sending mail with results at cod@fotonower.com refus_total : 0.24126676634409014 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21019805 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21019805_04-03-2025_18_22_17.pdf results_Qualipapia_P21019805_04-03-2025_18_22_17.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21019805_04-03-2025_18_22_17.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4234','21019805','results_Qualipapia_P21019805_04-03-2025_18_22_17.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21019805_04-03-2025_18_22_17.pdf','pdf','','0.14','0.24126676634409014') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1340835968, 1340835963] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2633619') ('4234', '21019805', '1340835968', None, None, None, None, None, '2633619') ('4234', None, None, None, None, None, None, None, '2633619') ('4234', '21019805', '1340835963', None, None, None, None, None, '2633619') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 2 time used for this insertion : 0.011475324630737305 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.217183351516724 time spend to save output : 0.011623859405517578 total time spend for step 10 : 4.228807210922241 step11:split_time_score Tue Mar 4 18:22:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 3442, 'mtr_user_id': 31, 'name': 'classifieur_2camions_valcor_021122_v1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'deux_camions,camion_droite,camion_gauche,pas_de_camion', 'svm_portfolios_learning': '7659379,7659034,7657685,7657114', 'photo_hashtag_type': 4458, 'photo_desc_type': 5723, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107760533,2107760534,2107760535,2107760536'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('15', 2),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 04032025 21019805 Nombre de photos uploadées : 2 / 23040 (0%) 04032025 21019805 Nombre de photos taguées (types de déchets): 2 / 2 (100%) 04032025 21019805 Nombre de photos taguées (volume) : 0 / 2 (0%) elapsed_time : load_data_split_time_score 2.6226043701171875e-06 elapsed_time : order_list_meta_photo_and_scores 7.867813110351562e-06 ?? elapsed_time : fill_and_build_computed_from_old_data 0.00020599365234375 elapsed_time : insert_dashboard_record_day_entry 0.022753000259399414 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.2379137549880031 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21007616_04-03-2025_18_02_31.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21007616 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21007616 AND mptpi.`type`=4230 To do Qualite : 0.2607389714178065 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21007617_04-03-2025_18_08_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21007617 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21007617 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21007619 order by id desc limit 1 Qualite : 0.23000331966758425 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21007621_04-03-2025_18_17_22.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21007621 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21007621 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21007623 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21007625 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21007627 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21007630 order by id desc limit 1 Qualite : 0.24892527834258063 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21007632_04-03-2025_12_47_08.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21007632 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21007632 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21008149 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21008150 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21008151 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21008152 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21008154 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21008156 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21008159 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21008161 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21008163 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21008165 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21008167 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21008169 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21008171 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21008174 order by id desc limit 1 Qualite : 0.2545628832014054 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21008176_04-03-2025_13_32_42.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21008176 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21008176 AND mptpi.`type`=4230 To do Qualite : 0.2882699738249937 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21008178_04-03-2025_13_22_26.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21008178 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21008178 AND mptpi.`type`=4230 To do Qualite : 0.2752652575950524 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21008180_04-03-2025_13_12_00.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21008180 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21008180 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21008682 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21008684 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21008686 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21008688 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21008690 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21008692 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21009813 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21009814 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21009816 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21009818 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21009820 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21012337 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21012338 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21012339 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21012340 order by id desc limit 1 Qualite : 0.2786803501749375 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21012341_04-03-2025_15_33_27.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21012341 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21012341 AND mptpi.`type`=4230 To do Qualite : 0.25563508816480707 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21012342_04-03-2025_15_17_16.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21012342 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21012342 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21012344 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21013583 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21013584 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21013596 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21013603 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21013621 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21013622 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21013623 order by id desc limit 1 Qualite : 0.2241280834880003 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21013624_04-03-2025_16_47_56.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21013624 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21013624 AND mptpi.`type`=4230 To do Qualite : 0.2287594422596334 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21013625_04-03-2025_16_41_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21013625 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21013625 AND mptpi.`type`=4230 To do Qualite : 0.25443374548275804 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21013627_04-03-2025_16_30_25.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21013627 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21013627 AND mptpi.`type`=4230 To do Qualite : 0.13308647995228717 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21013631_04-03-2025_16_02_10.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21013631 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21013631 AND mptpi.`type`=4230 To do Qualite : 0.2233305576348793 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21013634_04-03-2025_15_47_52.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21013634 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21013634 AND mptpi.`type`=4230 To do Qualite : 0.23128370903986467 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21017649_04-03-2025_17_43_27.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21017649 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21017649 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21017650 order by id desc limit 1 Qualite : 0.16063543629749746 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21017651_04-03-2025_17_17_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21017651 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21017651 AND mptpi.`type`=4230 To do Qualite : 0.22822148744495957 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21019034_04-03-2025_17_46_39.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21019034 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21019034 AND mptpi.`type`=4230 To do Qualite : 0.24126676634409014 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21019805_04-03-2025_18_22_17.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21019805 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21019805 AND mptpi.`type`=4230 To do Qualite : 0.15526997027984552 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21019806_04-03-2025_18_17_52.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21019806 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21019806 AND mptpi.`type`=4230 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'04032025': {'nb_upload': 2, 'nb_taggue_class': 2, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1340835968, 1340835963] Looping around the photos to save general results len do output : 1 /21019805Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2633619') ('4234', '21019805', '1340835968', None, None, None, None, None, '2633619') ('4234', None, None, None, None, None, None, None, '2633619') ('4234', '21019805', '1340835963', None, None, None, None, None, '2633619') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.012512445449829102 save_final save missing photos in datou_result : time spend for datou_step_exec : 34.681357860565186 time spend to save output : 0.012699127197265625 total time spend for step 11 : 34.69405698776245 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 1 len(input) + len(total_photo_id_missing) : 2 set_done_treatment 31.29user 23.18system 2:32.04elapsed 35%CPU (0avgtext+0avgdata 2355664maxresident)k 513128inputs+9576outputs (976major+1921738minor)pagefaults 0swaps