python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 4234 -l 5 ' -s datou_current_4234 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2958243 load datou : 4234 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load THCL from format json or kwargs add thcl : 3237 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig add pdt : 5561 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4234, datou_cur_ids : ['2589744'] with mtr_portfolio_ids : ['20578763'] and first list_photo_ids : [] new path : /proc/2958243/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, brightness, blur_detection, rle_unique_nms_with_priority, crop_condition, thcl, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , BFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 2 ; length of list_pids : 2 ; length of list_args : 2 time to download the photos : 0.24533462524414062 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Sat Feb 15 00:20:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 5691 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-15 00:20:30.925745: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-15 00:20:30.951175: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-15 00:20:30.953729: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f7b34000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-15 00:20:30.953798: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-15 00:20:30.957541: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-15 00:20:31.198430: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x35519960 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-15 00:20:31.198538: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-15 00:20:31.200048: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-15 00:20:31.200571: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-15 00:20:31.203789: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-15 00:20:31.206828: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-15 00:20:31.207380: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-15 00:20:31.209887: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-15 00:20:31.211026: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-15 00:20:31.215725: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-15 00:20:31.217001: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-15 00:20:31.217087: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-15 00:20:31.217760: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-15 00:20:31.217778: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-15 00:20:31.217788: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-15 00:20:31.218891: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 5207 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-15 00:20:31.498055: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-15 00:20:31.498161: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-15 00:20:31.498190: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-15 00:20:31.498215: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-15 00:20:31.498240: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-15 00:20:31.498264: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-15 00:20:31.498288: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-15 00:20:31.498313: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-15 00:20:31.500040: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-15 00:20:31.501418: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-15 00:20:31.501514: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-15 00:20:31.501531: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-15 00:20:31.501545: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-15 00:20:31.501560: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-15 00:20:31.501573: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-15 00:20:31.501587: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-15 00:20:31.501601: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-15 00:20:31.502611: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-15 00:20:31.502641: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-15 00:20:31.502650: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-15 00:20:31.502658: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-15 00:20:31.503752: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 5207 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-15 00:20:38.696907: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-15 00:20:38.898762: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-15 00:20:40.577104: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:40.577903: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.60G (3865470464 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.407619: W tensorflow/core/common_runtime/bfc_allocator.cc:311] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature. 2025-02-15 00:20:41.484357: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.484457: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.29GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-15 00:20:41.485808: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.485845: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.29GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-15 00:20:41.493006: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.493036: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.78GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-15 00:20:41.493796: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.493812: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.78GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-15 00:20:41.528712: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.528794: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 19.91MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-15 00:20:41.528831: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-02-15 00:20:41.530167: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.530208: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 16.00MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-15 00:20:41.531546: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.531576: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 16.00MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-15 00:20:41.539527: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.539588: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 63.85MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-15 00:20:41.540918: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.540943: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 63.85MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-15 00:20:41.542416: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.542446: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.26GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-15 00:20:41.544141: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.552519: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.553276: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.571848: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.572641: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.573394: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.574131: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.578444: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.579200: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.579926: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.580600: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.581693: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.592067: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.592829: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.602139: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.602852: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.603622: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.604314: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.605007: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-15 00:20:41.605687: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 2 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.74688 max: 150.40469 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 8.606910705566406e-05 nb_pixel_total : 863 time to create 1 rle with old method : 0.0016930103302001953 length of segment : 37 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 98 time to create 1 rle with old method : 0.00021123886108398438 length of segment : 36 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 115 time to create 1 rle with old method : 0.0002741813659667969 length of segment : 30 time for calcul the mask position with numpy : 4.9114227294921875e-05 nb_pixel_total : 50 time to create 1 rle with old method : 0.000110626220703125 length of segment : 25 time for calcul the mask position with numpy : 3.886222839355469e-05 nb_pixel_total : 89 time to create 1 rle with old method : 0.00019025802612304688 length of segment : 29 time for calcul the mask position with numpy : 5.459785461425781e-05 nb_pixel_total : 710 time to create 1 rle with old method : 0.0013363361358642578 length of segment : 31 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -116.83281 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 13 time for calcul the mask position with numpy : 0.00010633468627929688 nb_pixel_total : 1044 time to create 1 rle with old method : 0.0016942024230957031 length of segment : 69 time for calcul the mask position with numpy : 7.462501525878906e-05 nb_pixel_total : 1298 time to create 1 rle with old method : 0.0018475055694580078 length of segment : 82 time for calcul the mask position with numpy : 7.534027099609375e-05 nb_pixel_total : 29 time to create 1 rle with old method : 0.00011301040649414062 length of segment : 6 time for calcul the mask position with numpy : 6.937980651855469e-05 nb_pixel_total : 941 time to create 1 rle with old method : 0.0017824172973632812 length of segment : 42 time for calcul the mask position with numpy : 5.507469177246094e-05 nb_pixel_total : 685 time to create 1 rle with old method : 0.001058816909790039 length of segment : 37 time for calcul the mask position with numpy : 8.130073547363281e-05 nb_pixel_total : 960 time to create 1 rle with old method : 0.0019459724426269531 length of segment : 24 time for calcul the mask position with numpy : 4.696846008300781e-05 nb_pixel_total : 494 time to create 1 rle with old method : 0.0008668899536132812 length of segment : 16 time for calcul the mask position with numpy : 4.863739013671875e-05 nb_pixel_total : 640 time to create 1 rle with old method : 0.0009894371032714844 length of segment : 43 time for calcul the mask position with numpy : 3.981590270996094e-05 nb_pixel_total : 34 time to create 1 rle with old method : 9.989738464355469e-05 length of segment : 6 time for calcul the mask position with numpy : 5.054473876953125e-05 nb_pixel_total : 551 time to create 1 rle with old method : 0.0009679794311523438 length of segment : 17 time for calcul the mask position with numpy : 4.410743713378906e-05 nb_pixel_total : 32 time to create 1 rle with old method : 9.489059448242188e-05 length of segment : 6 time for calcul the mask position with numpy : 6.532669067382812e-05 nb_pixel_total : 1426 time to create 1 rle with old method : 0.002082347869873047 length of segment : 62 time for calcul the mask position with numpy : 5.6743621826171875e-05 nb_pixel_total : 559 time to create 1 rle with old method : 0.0009500980377197266 length of segment : 34 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -118.57109 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 14 time for calcul the mask position with numpy : 0.0001251697540283203 nb_pixel_total : 4368 time to create 1 rle with old method : 0.005460977554321289 length of segment : 121 time for calcul the mask position with numpy : 5.626678466796875e-05 nb_pixel_total : 542 time to create 1 rle with old method : 0.0010218620300292969 length of segment : 26 time for calcul the mask position with numpy : 0.00010156631469726562 nb_pixel_total : 1814 time to create 1 rle with old method : 0.0025746822357177734 length of segment : 53 time for calcul the mask position with numpy : 4.696846008300781e-05 nb_pixel_total : 392 time to create 1 rle with old method : 0.0006110668182373047 length of segment : 38 time for calcul the mask position with numpy : 8.034706115722656e-05 nb_pixel_total : 2194 time to create 1 rle with old method : 0.003057241439819336 length of segment : 34 time for calcul the mask position with numpy : 9.179115295410156e-05 nb_pixel_total : 1920 time to create 1 rle with old method : 0.0027451515197753906 length of segment : 72 time for calcul the mask position with numpy : 5.0067901611328125e-05 nb_pixel_total : 170 time to create 1 rle with old method : 0.0003781318664550781 length of segment : 10 time for calcul the mask position with numpy : 7.939338684082031e-05 nb_pixel_total : 1093 time to create 1 rle with old method : 0.002167940139770508 length of segment : 28 time for calcul the mask position with numpy : 4.673004150390625e-05 nb_pixel_total : 1050 time to create 1 rle with old method : 0.0014903545379638672 length of segment : 45 time for calcul the mask position with numpy : 3.695487976074219e-05 nb_pixel_total : 417 time to create 1 rle with old method : 0.0006046295166015625 length of segment : 22 time for calcul the mask position with numpy : 6.961822509765625e-05 nb_pixel_total : 1463 time to create 1 rle with old method : 0.0021026134490966797 length of segment : 36 time for calcul the mask position with numpy : 3.552436828613281e-05 nb_pixel_total : 42 time to create 1 rle with old method : 0.00012254714965820312 length of segment : 6 time for calcul the mask position with numpy : 0.000118255615234375 nb_pixel_total : 4126 time to create 1 rle with old method : 0.005670309066772461 length of segment : 39 time for calcul the mask position with numpy : 4.1484832763671875e-05 nb_pixel_total : 187 time to create 1 rle with old method : 0.0003578662872314453 length of segment : 10 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -112.46172 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 7.033348083496094e-05 nb_pixel_total : 1008 time to create 1 rle with old method : 0.0012936592102050781 length of segment : 78 time for calcul the mask position with numpy : 0.00020265579223632812 nb_pixel_total : 10309 time to create 1 rle with old method : 0.014181375503540039 length of segment : 135 time for calcul the mask position with numpy : 7.009506225585938e-05 nb_pixel_total : 520 time to create 1 rle with old method : 0.0007483959197998047 length of segment : 78 time for calcul the mask position with numpy : 4.601478576660156e-05 nb_pixel_total : 416 time to create 1 rle with old method : 0.0007660388946533203 length of segment : 23 time for calcul the mask position with numpy : 6.580352783203125e-05 nb_pixel_total : 1230 time to create 1 rle with old method : 0.0019779205322265625 length of segment : 76 time for calcul the mask position with numpy : 4.7206878662109375e-05 nb_pixel_total : 313 time to create 1 rle with old method : 0.0005235671997070312 length of segment : 21 time for calcul the mask position with numpy : 8.106231689453125e-05 nb_pixel_total : 1014 time to create 1 rle with old method : 0.0014734268188476562 length of segment : 96 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 131.25234 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 5.984306335449219e-05 nb_pixel_total : 547 time to create 1 rle with old method : 0.0009591579437255859 length of segment : 19 time for calcul the mask position with numpy : 6.151199340820312e-05 nb_pixel_total : 1423 time to create 1 rle with old method : 0.002108335494995117 length of segment : 150 time for calcul the mask position with numpy : 4.649162292480469e-05 nb_pixel_total : 139 time to create 1 rle with old method : 0.0002949237823486328 length of segment : 12 time for calcul the mask position with numpy : 9.560585021972656e-05 nb_pixel_total : 1786 time to create 1 rle with old method : 0.0026569366455078125 length of segment : 75 time for calcul the mask position with numpy : 4.887580871582031e-05 nb_pixel_total : 507 time to create 1 rle with old method : 0.0008258819580078125 length of segment : 19 Processing 1 images image shape: (400, 400, 3) min: 17.00000 max: 216.00000 molded_images shape: (1, 640, 640, 3) min: -82.51641 max: 85.74688 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -109.63750 max: 150.63125 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 7.367134094238281e-05 nb_pixel_total : 2673 time to create 1 rle with old method : 0.003570556640625 length of segment : 49 time for calcul the mask position with numpy : 0.00024509429931640625 nb_pixel_total : 10048 time to create 1 rle with old method : 0.012799739837646484 length of segment : 145 time for calcul the mask position with numpy : 6.270408630371094e-05 nb_pixel_total : 1266 time to create 1 rle with old method : 0.0017273426055908203 length of segment : 31 time for calcul the mask position with numpy : 4.9114227294921875e-05 nb_pixel_total : 409 time to create 1 rle with old method : 0.0006082057952880859 length of segment : 17 time for calcul the mask position with numpy : 4.982948303222656e-05 nb_pixel_total : 1386 time to create 1 rle with old method : 0.0020046234130859375 length of segment : 31 time for calcul the mask position with numpy : 4.00543212890625e-05 nb_pixel_total : 396 time to create 1 rle with old method : 0.0006451606750488281 length of segment : 17 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -110.03594 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 4.887580871582031e-05 nb_pixel_total : 1200 time to create 1 rle with old method : 0.0016202926635742188 length of segment : 28 time for calcul the mask position with numpy : 4.410743713378906e-05 nb_pixel_total : 821 time to create 1 rle with old method : 0.0010967254638671875 length of segment : 33 time for calcul the mask position with numpy : 4.649162292480469e-05 nb_pixel_total : 681 time to create 1 rle with old method : 0.0009074211120605469 length of segment : 51 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.20000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 10 time for calcul the mask position with numpy : 5.698204040527344e-05 nb_pixel_total : 56 time to create 1 rle with old method : 0.00012922286987304688 length of segment : 8 time for calcul the mask position with numpy : 5.435943603515625e-05 nb_pixel_total : 994 time to create 1 rle with old method : 0.0013701915740966797 length of segment : 26 time for calcul the mask position with numpy : 5.698204040527344e-05 nb_pixel_total : 874 time to create 1 rle with old method : 0.0012156963348388672 length of segment : 36 time for calcul the mask position with numpy : 0.00011277198791503906 nb_pixel_total : 1622 time to create 1 rle with old method : 0.002079010009765625 length of segment : 130 time for calcul the mask position with numpy : 5.7220458984375e-05 nb_pixel_total : 1103 time to create 1 rle with old method : 0.0014798641204833984 length of segment : 29 time for calcul the mask position with numpy : 3.838539123535156e-05 nb_pixel_total : 177 time to create 1 rle with old method : 0.0002770423889160156 length of segment : 18 time for calcul the mask position with numpy : 4.8160552978515625e-05 nb_pixel_total : 1255 time to create 1 rle with old method : 0.0015971660614013672 length of segment : 40 time for calcul the mask position with numpy : 4.76837158203125e-05 nb_pixel_total : 1285 time to create 1 rle with old method : 0.0016436576843261719 length of segment : 40 time for calcul the mask position with numpy : 0.00019741058349609375 nb_pixel_total : 3268 time to create 1 rle with old method : 0.004316806793212891 length of segment : 107 time for calcul the mask position with numpy : 5.125999450683594e-05 nb_pixel_total : 847 time to create 1 rle with old method : 0.0010781288146972656 length of segment : 39 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 135.72109 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 9.799003601074219e-05 nb_pixel_total : 1733 time to create 1 rle with old method : 0.0023190975189208984 length of segment : 187 time for calcul the mask position with numpy : 0.0002815723419189453 nb_pixel_total : 2115 time to create 1 rle with old method : 0.004712343215942383 length of segment : 143 time for calcul the mask position with numpy : 5.412101745605469e-05 nb_pixel_total : 259 time to create 1 rle with old method : 0.00040459632873535156 length of segment : 43 time for calcul the mask position with numpy : 7.295608520507812e-05 nb_pixel_total : 1408 time to create 1 rle with old method : 0.0018396377563476562 length of segment : 39 time for calcul the mask position with numpy : 7.200241088867188e-05 nb_pixel_total : 1625 time to create 1 rle with old method : 0.0020685195922851562 length of segment : 77 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 212.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 77.32500 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0009379386901855469 nb_pixel_total : 106798 time to create 1 rle with old method : 0.11683201789855957 length of segment : 280 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 11 time for calcul the mask position with numpy : 5.6743621826171875e-05 nb_pixel_total : 1084 time to create 1 rle with old method : 0.0015637874603271484 length of segment : 62 time for calcul the mask position with numpy : 8.916854858398438e-05 nb_pixel_total : 3594 time to create 1 rle with old method : 0.004510641098022461 length of segment : 60 time for calcul the mask position with numpy : 0.00010609626770019531 nb_pixel_total : 1601 time to create 1 rle with old method : 0.002296924591064453 length of segment : 49 time for calcul the mask position with numpy : 8.058547973632812e-05 nb_pixel_total : 1511 time to create 1 rle with old method : 0.0020058155059814453 length of segment : 38 time for calcul the mask position with numpy : 5.316734313964844e-05 nb_pixel_total : 885 time to create 1 rle with old method : 0.0012264251708984375 length of segment : 26 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 169 time to create 1 rle with old method : 0.0002796649932861328 length of segment : 16 time for calcul the mask position with numpy : 4.482269287109375e-05 nb_pixel_total : 420 time to create 1 rle with old method : 0.0006301403045654297 length of segment : 26 time for calcul the mask position with numpy : 9.298324584960938e-05 nb_pixel_total : 1690 time to create 1 rle with old method : 0.0022573471069335938 length of segment : 64 time for calcul the mask position with numpy : 6.103515625e-05 nb_pixel_total : 641 time to create 1 rle with old method : 0.0009748935699462891 length of segment : 30 time for calcul the mask position with numpy : 6.961822509765625e-05 nb_pixel_total : 1715 time to create 1 rle with old method : 0.0023059844970703125 length of segment : 78 time for calcul the mask position with numpy : 7.557868957519531e-05 nb_pixel_total : 1636 time to create 1 rle with old method : 0.0021469593048095703 length of segment : 72 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.88516 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 20 time for calcul the mask position with numpy : 6.175041198730469e-05 nb_pixel_total : 337 time to create 1 rle with old method : 0.0004622936248779297 length of segment : 21 time for calcul the mask position with numpy : 5.340576171875e-05 nb_pixel_total : 589 time to create 1 rle with old method : 0.0008327960968017578 length of segment : 39 time for calcul the mask position with numpy : 5.030632019042969e-05 nb_pixel_total : 366 time to create 1 rle with old method : 0.0005235671997070312 length of segment : 29 time for calcul the mask position with numpy : 4.00543212890625e-05 nb_pixel_total : 199 time to create 1 rle with old method : 0.0003058910369873047 length of segment : 15 time for calcul the mask position with numpy : 4.291534423828125e-05 nb_pixel_total : 167 time to create 1 rle with old method : 0.0002779960632324219 length of segment : 24 time for calcul the mask position with numpy : 4.887580871582031e-05 nb_pixel_total : 355 time to create 1 rle with old method : 0.0005362033843994141 length of segment : 24 time for calcul the mask position with numpy : 5.1021575927734375e-05 nb_pixel_total : 364 time to create 1 rle with old method : 0.0004980564117431641 length of segment : 41 time for calcul the mask position with numpy : 3.838539123535156e-05 nb_pixel_total : 130 time to create 1 rle with old method : 0.00021910667419433594 length of segment : 11 time for calcul the mask position with numpy : 3.743171691894531e-05 nb_pixel_total : 80 time to create 1 rle with old method : 0.00015616416931152344 length of segment : 10 time for calcul the mask position with numpy : 4.172325134277344e-05 nb_pixel_total : 202 time to create 1 rle with old method : 0.0003223419189453125 length of segment : 26 time for calcul the mask position with numpy : 4.2438507080078125e-05 nb_pixel_total : 248 time to create 1 rle with old method : 0.00041556358337402344 length of segment : 23 time for calcul the mask position with numpy : 6.4849853515625e-05 nb_pixel_total : 407 time to create 1 rle with old method : 0.0006265640258789062 length of segment : 43 time for calcul the mask position with numpy : 4.100799560546875e-05 nb_pixel_total : 156 time to create 1 rle with old method : 0.00025153160095214844 length of segment : 14 time for calcul the mask position with numpy : 3.743171691894531e-05 nb_pixel_total : 143 time to create 1 rle with old method : 0.00023031234741210938 length of segment : 15 time for calcul the mask position with numpy : 6.103515625e-05 nb_pixel_total : 896 time to create 1 rle with old method : 0.0011582374572753906 length of segment : 43 time for calcul the mask position with numpy : 7.653236389160156e-05 nb_pixel_total : 516 time to create 1 rle with old method : 0.0013599395751953125 length of segment : 23 time for calcul the mask position with numpy : 4.100799560546875e-05 nb_pixel_total : 104 time to create 1 rle with old method : 0.00018668174743652344 length of segment : 12 time for calcul the mask position with numpy : 3.3855438232421875e-05 nb_pixel_total : 124 time to create 1 rle with old method : 0.0001995563507080078 length of segment : 14 time for calcul the mask position with numpy : 3.123283386230469e-05 nb_pixel_total : 104 time to create 1 rle with old method : 0.00017380714416503906 length of segment : 12 time for calcul the mask position with numpy : 3.4332275390625e-05 nb_pixel_total : 200 time to create 1 rle with old method : 0.00030112266540527344 length of segment : 16 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 149.33437 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 6.008148193359375e-05 nb_pixel_total : 570 time to create 1 rle with old method : 0.0009262561798095703 length of segment : 34 time for calcul the mask position with numpy : 5.340576171875e-05 nb_pixel_total : 211 time to create 1 rle with old method : 0.00043582916259765625 length of segment : 13 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 222 time to create 1 rle with old method : 0.00040149688720703125 length of segment : 23 time for calcul the mask position with numpy : 5.2928924560546875e-05 nb_pixel_total : 101 time to create 1 rle with old method : 0.00017571449279785156 length of segment : 24 time for calcul the mask position with numpy : 3.8623809814453125e-05 nb_pixel_total : 175 time to create 1 rle with old method : 0.0002989768981933594 length of segment : 20 time for calcul the mask position with numpy : 6.4849853515625e-05 nb_pixel_total : 185 time to create 1 rle with old method : 0.00043582916259765625 length of segment : 59 time for calcul the mask position with numpy : 5.340576171875e-05 nb_pixel_total : 383 time to create 1 rle with old method : 0.0006718635559082031 length of segment : 30 time for calcul the mask position with numpy : 5.125999450683594e-05 nb_pixel_total : 291 time to create 1 rle with old method : 0.00043463706970214844 length of segment : 23 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 134.91250 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 6.29425048828125e-05 nb_pixel_total : 15 time to create 1 rle with old method : 9.1552734375e-05 length of segment : 3 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.93047 max: 148.00234 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 6.031990051269531e-05 nb_pixel_total : 873 time to create 1 rle with old method : 0.0014793872833251953 length of segment : 33 time for calcul the mask position with numpy : 4.5299530029296875e-05 nb_pixel_total : 97 time to create 1 rle with old method : 0.00020265579223632812 length of segment : 36 time for calcul the mask position with numpy : 3.4332275390625e-05 nb_pixel_total : 120 time to create 1 rle with old method : 0.00019884109497070312 length of segment : 31 time for calcul the mask position with numpy : 0.00013017654418945312 nb_pixel_total : 4295 time to create 1 rle with old method : 0.005672454833984375 length of segment : 67 time for calcul the mask position with numpy : 0.0001468658447265625 nb_pixel_total : 3784 time to create 1 rle with old method : 0.004770755767822266 length of segment : 64 time for calcul the mask position with numpy : 5.5789947509765625e-05 nb_pixel_total : 659 time to create 1 rle with old method : 0.0009405612945556641 length of segment : 28 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.51250 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 4.744529724121094e-05 nb_pixel_total : 600 time to create 1 rle with old method : 0.0008237361907958984 length of segment : 33 time for calcul the mask position with numpy : 7.271766662597656e-05 nb_pixel_total : 1448 time to create 1 rle with old method : 0.0018773078918457031 length of segment : 79 time for calcul the mask position with numpy : 4.172325134277344e-05 nb_pixel_total : 522 time to create 1 rle with old method : 0.0008187294006347656 length of segment : 17 time for calcul the mask position with numpy : 4.935264587402344e-05 nb_pixel_total : 1124 time to create 1 rle with old method : 0.001470327377319336 length of segment : 37 time for calcul the mask position with numpy : 6.580352783203125e-05 nb_pixel_total : 1007 time to create 1 rle with old method : 0.0013461112976074219 length of segment : 80 time for calcul the mask position with numpy : 8.130073547363281e-05 nb_pixel_total : 1574 time to create 1 rle with old method : 0.0021314620971679688 length of segment : 66 time for calcul the mask position with numpy : 4.9114227294921875e-05 nb_pixel_total : 357 time to create 1 rle with old method : 0.0005462169647216797 length of segment : 25 time for calcul the mask position with numpy : 3.910064697265625e-05 nb_pixel_total : 46 time to create 1 rle with old method : 0.00010395050048828125 length of segment : 7 time for calcul the mask position with numpy : 4.220008850097656e-05 nb_pixel_total : 426 time to create 1 rle with old method : 0.0006170272827148438 length of segment : 38 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -119.07891 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 11 time for calcul the mask position with numpy : 0.0001277923583984375 nb_pixel_total : 4807 time to create 1 rle with old method : 0.006039857864379883 length of segment : 142 time for calcul the mask position with numpy : 6.198883056640625e-05 nb_pixel_total : 348 time to create 1 rle with old method : 0.0005373954772949219 length of segment : 29 time for calcul the mask position with numpy : 3.528594970703125e-05 nb_pixel_total : 34 time to create 1 rle with old method : 0.00010538101196289062 length of segment : 6 time for calcul the mask position with numpy : 3.7670135498046875e-05 nb_pixel_total : 302 time to create 1 rle with old method : 0.00045299530029296875 length of segment : 16 time for calcul the mask position with numpy : 3.2901763916015625e-05 nb_pixel_total : 183 time to create 1 rle with old method : 0.00028896331787109375 length of segment : 11 time for calcul the mask position with numpy : 3.504753112792969e-05 nb_pixel_total : 342 time to create 1 rle with old method : 0.0005247592926025391 length of segment : 23 time for calcul the mask position with numpy : 3.218650817871094e-05 nb_pixel_total : 221 time to create 1 rle with old method : 0.0003497600555419922 length of segment : 11 time for calcul the mask position with numpy : 5.6743621826171875e-05 nb_pixel_total : 1454 time to create 1 rle with old method : 0.0019876956939697266 length of segment : 28 time for calcul the mask position with numpy : 9.655952453613281e-05 nb_pixel_total : 1254 time to create 1 rle with old method : 0.0017693042755126953 length of segment : 40 time for calcul the mask position with numpy : 5.8650970458984375e-05 nb_pixel_total : 1180 time to create 1 rle with old method : 0.0016295909881591797 length of segment : 38 time for calcul the mask position with numpy : 6.318092346191406e-05 nb_pixel_total : 1630 time to create 1 rle with old method : 0.002179384231567383 length of segment : 31 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -106.89531 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 9.989738464355469e-05 nb_pixel_total : 223 time to create 1 rle with old method : 0.0003466606140136719 length of segment : 63 time for calcul the mask position with numpy : 0.00022983551025390625 nb_pixel_total : 11035 time to create 1 rle with old method : 0.01435399055480957 length of segment : 134 time for calcul the mask position with numpy : 5.030632019042969e-05 nb_pixel_total : 334 time to create 1 rle with old method : 0.0004992485046386719 length of segment : 33 time for calcul the mask position with numpy : 4.1961669921875e-05 nb_pixel_total : 203 time to create 1 rle with old method : 0.00028014183044433594 length of segment : 31 time for calcul the mask position with numpy : 4.6253204345703125e-05 nb_pixel_total : 868 time to create 1 rle with old method : 0.0011210441589355469 length of segment : 59 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 417 time to create 1 rle with old method : 0.0006849765777587891 length of segment : 23 time for calcul the mask position with numpy : 3.528594970703125e-05 nb_pixel_total : 302 time to create 1 rle with old method : 0.0004286766052246094 length of segment : 19 time for calcul the mask position with numpy : 3.218650817871094e-05 nb_pixel_total : 225 time to create 1 rle with old method : 0.0003452301025390625 length of segment : 17 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 133.59219 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 8.893013000488281e-05 nb_pixel_total : 1982 time to create 1 rle with old method : 0.002270221710205078 length of segment : 169 time for calcul the mask position with numpy : 6.4849853515625e-05 nb_pixel_total : 795 time to create 1 rle with old method : 0.0010838508605957031 length of segment : 47 time for calcul the mask position with numpy : 4.744529724121094e-05 nb_pixel_total : 432 time to create 1 rle with old method : 0.0006382465362548828 length of segment : 17 time for calcul the mask position with numpy : 5.817413330078125e-05 nb_pixel_total : 1453 time to create 1 rle with old method : 0.0019145011901855469 length of segment : 39 time for calcul the mask position with numpy : 7.772445678710938e-05 nb_pixel_total : 2185 time to create 1 rle with old method : 0.0028493404388427734 length of segment : 57 Processing 1 images image shape: (400, 400, 3) min: 16.00000 max: 212.00000 molded_images shape: (1, 640, 640, 3) min: -86.37578 max: 81.74688 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -110.57500 max: 151.04531 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 0.0001537799835205078 nb_pixel_total : 9036 time to create 1 rle with old method : 0.01030278205871582 length of segment : 165 time for calcul the mask position with numpy : 0.00010609626770019531 nb_pixel_total : 2635 time to create 1 rle with old method : 0.0032727718353271484 length of segment : 48 time for calcul the mask position with numpy : 5.745887756347656e-05 nb_pixel_total : 1376 time to create 1 rle with old method : 0.0017924308776855469 length of segment : 31 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -119.43438 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 10 time for calcul the mask position with numpy : 5.364418029785156e-05 nb_pixel_total : 872 time to create 1 rle with old method : 0.001094818115234375 length of segment : 67 time for calcul the mask position with numpy : 6.127357482910156e-05 nb_pixel_total : 1368 time to create 1 rle with old method : 0.0018472671508789062 length of segment : 45 time for calcul the mask position with numpy : 4.1484832763671875e-05 nb_pixel_total : 419 time to create 1 rle with old method : 0.0005853176116943359 length of segment : 31 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 834 time to create 1 rle with old method : 0.001184225082397461 length of segment : 25 time for calcul the mask position with numpy : 3.4332275390625e-05 nb_pixel_total : 235 time to create 1 rle with old method : 0.00031876564025878906 length of segment : 30 time for calcul the mask position with numpy : 4.38690185546875e-05 nb_pixel_total : 766 time to create 1 rle with old method : 0.0015683174133300781 length of segment : 37 time for calcul the mask position with numpy : 3.62396240234375e-05 nb_pixel_total : 69 time to create 1 rle with old method : 0.00013065338134765625 length of segment : 12 time for calcul the mask position with numpy : 7.295608520507812e-05 nb_pixel_total : 858 time to create 1 rle with old method : 0.0012259483337402344 length of segment : 28 time for calcul the mask position with numpy : 4.673004150390625e-05 nb_pixel_total : 282 time to create 1 rle with old method : 0.0004565715789794922 length of segment : 21 time for calcul the mask position with numpy : 3.361701965332031e-05 nb_pixel_total : 91 time to create 1 rle with old method : 0.00017118453979492188 length of segment : 10 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -116.99688 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 11 time for calcul the mask position with numpy : 0.00018167495727539062 nb_pixel_total : 1814 time to create 1 rle with old method : 0.0038604736328125 length of segment : 134 time for calcul the mask position with numpy : 7.319450378417969e-05 nb_pixel_total : 965 time to create 1 rle with old method : 0.0017099380493164062 length of segment : 27 time for calcul the mask position with numpy : 5.7220458984375e-05 nb_pixel_total : 1242 time to create 1 rle with old method : 0.0015764236450195312 length of segment : 36 time for calcul the mask position with numpy : 5.745887756347656e-05 nb_pixel_total : 1104 time to create 1 rle with old method : 0.001466512680053711 length of segment : 39 time for calcul the mask position with numpy : 0.00026345252990722656 nb_pixel_total : 741 time to create 1 rle with old method : 0.0015404224395751953 length of segment : 62 time for calcul the mask position with numpy : 7.295608520507812e-05 nb_pixel_total : 956 time to create 1 rle with old method : 0.002105236053466797 length of segment : 37 time for calcul the mask position with numpy : 6.413459777832031e-05 nb_pixel_total : 72 time to create 1 rle with old method : 0.0002586841583251953 length of segment : 12 time for calcul the mask position with numpy : 5.078315734863281e-05 nb_pixel_total : 1003 time to create 1 rle with old method : 0.0013833045959472656 length of segment : 26 time for calcul the mask position with numpy : 5.078315734863281e-05 nb_pixel_total : 1236 time to create 1 rle with old method : 0.001589059829711914 length of segment : 41 time for calcul the mask position with numpy : 5.817413330078125e-05 nb_pixel_total : 818 time to create 1 rle with old method : 0.0014011859893798828 length of segment : 38 time for calcul the mask position with numpy : 6.556510925292969e-05 nb_pixel_total : 819 time to create 1 rle with old method : 0.001447916030883789 length of segment : 37 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 133.90859 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 6.318092346191406e-05 nb_pixel_total : 1426 time to create 1 rle with old method : 0.0018603801727294922 length of segment : 39 time for calcul the mask position with numpy : 0.0001499652862548828 nb_pixel_total : 1352 time to create 1 rle with old method : 0.0020308494567871094 length of segment : 224 time for calcul the mask position with numpy : 0.000194549560546875 nb_pixel_total : 2315 time to create 1 rle with old method : 0.0029633045196533203 length of segment : 162 time for calcul the mask position with numpy : 3.5762786865234375e-05 nb_pixel_total : 99 time to create 1 rle with old method : 0.00015544891357421875 length of segment : 21 time for calcul the mask position with numpy : 5.221366882324219e-05 nb_pixel_total : 734 time to create 1 rle with old method : 0.0009627342224121094 length of segment : 41 time for calcul the mask position with numpy : 5.507469177246094e-05 nb_pixel_total : 1482 time to create 1 rle with old method : 0.0020749568939208984 length of segment : 40 time for calcul the mask position with numpy : 7.176399230957031e-05 nb_pixel_total : 1536 time to create 1 rle with old method : 0.002071857452392578 length of segment : 80 time for calcul the mask position with numpy : 6.318092346191406e-05 nb_pixel_total : 1526 time to create 1 rle with old method : 0.0019419193267822266 length of segment : 73 time for calcul the mask position with numpy : 0.00020813941955566406 nb_pixel_total : 3603 time to create 1 rle with old method : 0.004549503326416016 length of segment : 205 Processing 1 images image shape: (280, 400, 3) min: 8.00000 max: 215.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 77.32500 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0009560585021972656 nb_pixel_total : 106818 time to create 1 rle with old method : 0.12977910041809082 length of segment : 282 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 7.915496826171875e-05 nb_pixel_total : 3682 time to create 1 rle with old method : 0.005169868469238281 length of segment : 62 time for calcul the mask position with numpy : 6.079673767089844e-05 nb_pixel_total : 827 time to create 1 rle with old method : 0.0010938644409179688 length of segment : 45 time for calcul the mask position with numpy : 9.870529174804688e-05 nb_pixel_total : 1716 time to create 1 rle with old method : 0.0022122859954833984 length of segment : 49 time for calcul the mask position with numpy : 6.29425048828125e-05 nb_pixel_total : 1591 time to create 1 rle with old method : 0.0019986629486083984 length of segment : 69 time for calcul the mask position with numpy : 3.361701965332031e-05 nb_pixel_total : 116 time to create 1 rle with old method : 0.00022649765014648438 length of segment : 14 time for calcul the mask position with numpy : 5.5789947509765625e-05 nb_pixel_total : 1618 time to create 1 rle with old method : 0.0019769668579101562 length of segment : 34 time for calcul the mask position with numpy : 4.291534423828125e-05 nb_pixel_total : 649 time to create 1 rle with old method : 0.0008881092071533203 length of segment : 30 time for calcul the mask position with numpy : 4.172325134277344e-05 nb_pixel_total : 817 time to create 1 rle with old method : 0.0011188983917236328 length of segment : 26 time for calcul the mask position with numpy : 3.790855407714844e-05 nb_pixel_total : 413 time to create 1 rle with old method : 0.0005609989166259766 length of segment : 29 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.42031 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 13 time for calcul the mask position with numpy : 4.863739013671875e-05 nb_pixel_total : 356 time to create 1 rle with old method : 0.0005509853363037109 length of segment : 21 time for calcul the mask position with numpy : 3.24249267578125e-05 nb_pixel_total : 99 time to create 1 rle with old method : 0.00019097328186035156 length of segment : 11 time for calcul the mask position with numpy : 3.0279159545898438e-05 nb_pixel_total : 127 time to create 1 rle with old method : 0.00020313262939453125 length of segment : 14 time for calcul the mask position with numpy : 3.075599670410156e-05 nb_pixel_total : 161 time to create 1 rle with old method : 0.0002560615539550781 length of segment : 15 time for calcul the mask position with numpy : 3.075599670410156e-05 nb_pixel_total : 87 time to create 1 rle with old method : 0.00015044212341308594 length of segment : 10 time for calcul the mask position with numpy : 3.4332275390625e-05 nb_pixel_total : 192 time to create 1 rle with old method : 0.00029015541076660156 length of segment : 25 time for calcul the mask position with numpy : 3.0517578125e-05 nb_pixel_total : 142 time to create 1 rle with old method : 0.000244140625 length of segment : 13 time for calcul the mask position with numpy : 3.647804260253906e-05 nb_pixel_total : 396 time to create 1 rle with old method : 0.0005335807800292969 length of segment : 29 time for calcul the mask position with numpy : 2.956390380859375e-05 nb_pixel_total : 116 time to create 1 rle with old method : 0.0001800060272216797 length of segment : 15 time for calcul the mask position with numpy : 3.0517578125e-05 nb_pixel_total : 148 time to create 1 rle with old method : 0.00022363662719726562 length of segment : 17 time for calcul the mask position with numpy : 3.743171691894531e-05 nb_pixel_total : 475 time to create 1 rle with old method : 0.0006766319274902344 length of segment : 36 time for calcul the mask position with numpy : 3.147125244140625e-05 nb_pixel_total : 162 time to create 1 rle with old method : 0.00025081634521484375 length of segment : 17 time for calcul the mask position with numpy : 3.647804260253906e-05 nb_pixel_total : 352 time to create 1 rle with old method : 0.00047588348388671875 length of segment : 25 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 145.19375 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 569 time to create 1 rle with old method : 0.0007414817810058594 length of segment : 28 time for calcul the mask position with numpy : 4.291534423828125e-05 nb_pixel_total : 130 time to create 1 rle with old method : 0.0002071857452392578 length of segment : 30 time for calcul the mask position with numpy : 4.1961669921875e-05 nb_pixel_total : 506 time to create 1 rle with old method : 0.0007302761077880859 length of segment : 33 time for calcul the mask position with numpy : 3.266334533691406e-05 nb_pixel_total : 152 time to create 1 rle with old method : 0.00024628639221191406 length of segment : 11 time for calcul the mask position with numpy : 3.24249267578125e-05 nb_pixel_total : 194 time to create 1 rle with old method : 0.00027942657470703125 length of segment : 21 time for calcul the mask position with numpy : 3.0517578125e-05 nb_pixel_total : 166 time to create 1 rle with old method : 0.00023746490478515625 length of segment : 22 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 134.91250 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 7.653236389160156e-05 nb_pixel_total : 274 time to create 1 rle with old method : 0.0005691051483154297 length of segment : 55 time for calcul the mask position with numpy : 3.0279159545898438e-05 nb_pixel_total : 70 time to create 1 rle with old method : 0.00011992454528808594 length of segment : 12 Detection mask done ! Trying to reset tf kernel 2958773 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 2504 tf kernel not reseted sub process len(results) : 213 len(list_Values) 213 None max_time_sub_proc : 3600 parent process len(results) : 0 len(list_Values) 213 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 3697 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Catched exception ! Connect or reconnect ! Number saved : None batch 1 Loaded 213 chid ids of type : 4228 Number RLEs to save : 9551 save missing photos in datou_result : time spend for datou_step_exec : 22.703126430511475 time spend to save output : 1.175854206085205 total time spend for step 1 : 23.87898063659668 step2:brightness Sat Feb 15 00:20:52 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1739575227_2958243_1337616822_9294d8bedba544ee103f34da10d7a7cd.jpg treat image : temp/1739575227_2958243_1337616763_18005def568dc56eb60db957ec5e76ad.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 2 time used for this insertion : 0.011113643646240234 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 2 time used for this insertion : 0.015180826187133789 save missing photos in datou_result : time spend for datou_step_exec : 2.1835594177246094 time spend to save output : 0.03081655502319336 total time spend for step 2 : 2.2143759727478027 step3:blur_detection Sat Feb 15 00:20:54 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1739575227_2958243_1337616822_9294d8bedba544ee103f34da10d7a7cd.jpg resize: (1080, 1920) 1337616822 -7.959349608786445 treat image : temp/1739575227_2958243_1337616763_18005def568dc56eb60db957ec5e76ad.jpg resize: (1080, 1920) 1337616763 -7.959972801904531 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 2 time used for this insertion : 0.017750024795532227 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 2 time used for this insertion : 0.016159772872924805 save missing photos in datou_result : time spend for datou_step_exec : 2.229703903198242 time spend to save output : 0.038118839263916016 total time spend for step 3 : 2.267822742462158 step4:rle_unique_nms_with_priority Sat Feb 15 00:20:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 213 chid ids of type : 4228 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 110 nb_hashtags : 7 time to prepare the origin masks : 1.0275113582611084 time for calcul the mask position with numpy : 0.0909736156463623 nb_pixel_total : 1870929 time to create 1 rle with new method : 0.4504544734954834 time for calcul the mask position with numpy : 0.008665800094604492 nb_pixel_total : 494 time to create 1 rle with old method : 0.0006079673767089844 time for calcul the mask position with numpy : 0.008718490600585938 nb_pixel_total : 59 time to create 1 rle with old method : 0.00011205673217773438 time for calcul the mask position with numpy : 0.008489370346069336 nb_pixel_total : 417 time to create 1 rle with old method : 0.0005125999450683594 time for calcul the mask position with numpy : 0.008490324020385742 nb_pixel_total : 20 time to create 1 rle with old method : 5.1021575927734375e-05 time for calcul the mask position with numpy : 0.008458852767944336 nb_pixel_total : 82 time to create 1 rle with old method : 0.00012350082397460938 time for calcul the mask position with numpy : 0.008478164672851562 nb_pixel_total : 392 time to create 1 rle with old method : 0.00047659873962402344 time for calcul the mask position with numpy : 0.008437871932983398 nb_pixel_total : 520 time to create 1 rle with old method : 0.0006120204925537109 time for calcul the mask position with numpy : 0.008440732955932617 nb_pixel_total : 1423 time to create 1 rle with old method : 0.0016796588897705078 time for calcul the mask position with numpy : 0.008497476577758789 nb_pixel_total : 1093 time to create 1 rle with old method : 0.0012712478637695312 time for calcul the mask position with numpy : 0.008480072021484375 nb_pixel_total : 50 time to create 1 rle with old method : 6.437301635742188e-05 time for calcul the mask position with numpy : 0.008517026901245117 nb_pixel_total : 27 time to create 1 rle with old method : 4.6253204345703125e-05 time for calcul the mask position with numpy : 0.008511543273925781 nb_pixel_total : 863 time to create 1 rle with old method : 0.0010366439819335938 time for calcul the mask position with numpy : 0.008536815643310547 nb_pixel_total : 98 time to create 1 rle with old method : 0.00012564659118652344 time for calcul the mask position with numpy : 0.008511066436767578 nb_pixel_total : 68 time to create 1 rle with old method : 0.00011610984802246094 time for calcul the mask position with numpy : 0.008461236953735352 nb_pixel_total : 115 time to create 1 rle with old method : 0.00015044212341308594 time for calcul the mask position with numpy : 0.008449554443359375 nb_pixel_total : 685 time to create 1 rle with old method : 0.0008063316345214844 time for calcul the mask position with numpy : 0.008288383483886719 nb_pixel_total : 7 time to create 1 rle with old method : 6.103515625e-05 time for calcul the mask position with numpy : 0.008444786071777344 nb_pixel_total : 34 time to create 1 rle with old method : 5.269050598144531e-05 time for calcul the mask position with numpy : 0.008480072021484375 nb_pixel_total : 32 time to create 1 rle with old method : 5.1021575927734375e-05 time for calcul the mask position with numpy : 0.00845193862915039 nb_pixel_total : 25 time to create 1 rle with old method : 6.270408630371094e-05 time for calcul the mask position with numpy : 0.008571863174438477 nb_pixel_total : 42 time to create 1 rle with old method : 7.295608520507812e-05 time for calcul the mask position with numpy : 0.008352518081665039 nb_pixel_total : 29 time to create 1 rle with old method : 4.7206878662109375e-05 time for calcul the mask position with numpy : 0.008527755737304688 nb_pixel_total : 1008 time to create 1 rle with old method : 0.0012145042419433594 time for calcul the mask position with numpy : 0.008512258529663086 nb_pixel_total : 108 time to create 1 rle with old method : 0.00015091896057128906 time for calcul the mask position with numpy : 0.008486270904541016 nb_pixel_total : 960 time to create 1 rle with old method : 0.0011382102966308594 time for calcul the mask position with numpy : 0.008452892303466797 nb_pixel_total : 1230 time to create 1 rle with old method : 0.0014507770538330078 time for calcul the mask position with numpy : 0.00844717025756836 nb_pixel_total : 296 time to create 1 rle with old method : 0.00041556358337402344 time for calcul the mask position with numpy : 0.008485794067382812 nb_pixel_total : 1426 time to create 1 rle with old method : 0.0015902519226074219 time for calcul the mask position with numpy : 0.008512258529663086 nb_pixel_total : 1044 time to create 1 rle with old method : 0.0011954307556152344 time for calcul the mask position with numpy : 0.008494853973388672 nb_pixel_total : 50 time to create 1 rle with old method : 7.748603820800781e-05 time for calcul the mask position with numpy : 0.008426904678344727 nb_pixel_total : 941 time to create 1 rle with old method : 0.0010836124420166016 time for calcul the mask position with numpy : 0.008466958999633789 nb_pixel_total : 2719 time to create 1 rle with old method : 0.003184795379638672 time for calcul the mask position with numpy : 0.008451223373413086 nb_pixel_total : 1463 time to create 1 rle with old method : 0.001682281494140625 time for calcul the mask position with numpy : 0.00846552848815918 nb_pixel_total : 416 time to create 1 rle with old method : 0.0005116462707519531 time for calcul the mask position with numpy : 0.008453845977783203 nb_pixel_total : 2194 time to create 1 rle with old method : 0.002554178237915039 time for calcul the mask position with numpy : 0.008442163467407227 nb_pixel_total : 4368 time to create 1 rle with old method : 0.0050165653228759766 time for calcul the mask position with numpy : 0.008436203002929688 nb_pixel_total : 182 time to create 1 rle with old method : 0.00027251243591308594 time for calcul the mask position with numpy : 0.008466482162475586 nb_pixel_total : 1814 time to create 1 rle with old method : 0.0021522045135498047 time for calcul the mask position with numpy : 0.00855708122253418 nb_pixel_total : 10309 time to create 1 rle with old method : 0.011563539505004883 time for calcul the mask position with numpy : 0.008483171463012695 nb_pixel_total : 139 time to create 1 rle with old method : 0.00017499923706054688 time for calcul the mask position with numpy : 0.00852656364440918 nb_pixel_total : 1786 time to create 1 rle with old method : 0.002045869827270508 time for calcul the mask position with numpy : 0.008492708206176758 nb_pixel_total : 542 time to create 1 rle with old method : 0.0006620883941650391 time for calcul the mask position with numpy : 0.008478879928588867 nb_pixel_total : 1050 time to create 1 rle with old method : 0.0012359619140625 time for calcul the mask position with numpy : 0.008481502532958984 nb_pixel_total : 15 time to create 1 rle with old method : 0.0001239776611328125 time for calcul the mask position with numpy : 0.008278369903564453 nb_pixel_total : 547 time to create 1 rle with old method : 0.000614166259765625 time for calcul the mask position with numpy : 0.008227825164794922 nb_pixel_total : 821 time to create 1 rle with old method : 0.0009453296661376953 time for calcul the mask position with numpy : 0.008422613143920898 nb_pixel_total : 3268 time to create 1 rle with old method : 0.0036902427673339844 time for calcul the mask position with numpy : 0.009488582611083984 nb_pixel_total : 1200 time to create 1 rle with old method : 0.001758575439453125 time for calcul the mask position with numpy : 0.008570671081542969 nb_pixel_total : 847 time to create 1 rle with old method : 0.0010585784912109375 time for calcul the mask position with numpy : 0.008445501327514648 nb_pixel_total : 68 time to create 1 rle with old method : 0.00011491775512695312 time for calcul the mask position with numpy : 0.008445978164672852 nb_pixel_total : 1255 time to create 1 rle with old method : 0.00142669677734375 time for calcul the mask position with numpy : 0.008446931838989258 nb_pixel_total : 1408 time to create 1 rle with old method : 0.0015795230865478516 time for calcul the mask position with numpy : 0.008468389511108398 nb_pixel_total : 10048 time to create 1 rle with old method : 0.011308431625366211 time for calcul the mask position with numpy : 0.008522987365722656 nb_pixel_total : 177 time to create 1 rle with old method : 0.0002193450927734375 time for calcul the mask position with numpy : 0.009582757949829102 nb_pixel_total : 259 time to create 1 rle with old method : 0.00038695335388183594 time for calcul the mask position with numpy : 0.008665323257446289 nb_pixel_total : 1622 time to create 1 rle with old method : 0.0019054412841796875 time for calcul the mask position with numpy : 0.008327245712280273 nb_pixel_total : 2115 time to create 1 rle with old method : 0.0024585723876953125 time for calcul the mask position with numpy : 0.008958578109741211 nb_pixel_total : 874 time to create 1 rle with old method : 0.0014312267303466797 time for calcul the mask position with numpy : 0.009892702102661133 nb_pixel_total : 681 time to create 1 rle with old method : 0.0011034011840820312 time for calcul the mask position with numpy : 0.009935379028320312 nb_pixel_total : 1733 time to create 1 rle with old method : 0.003704547882080078 time for calcul the mask position with numpy : 0.012369632720947266 nb_pixel_total : 2673 time to create 1 rle with old method : 0.003993988037109375 time for calcul the mask position with numpy : 0.008722066879272461 nb_pixel_total : 1625 time to create 1 rle with old method : 0.0019407272338867188 time for calcul the mask position with numpy : 0.008619070053100586 nb_pixel_total : 409 time to create 1 rle with old method : 0.0005257129669189453 time for calcul the mask position with numpy : 0.009572505950927734 nb_pixel_total : 4 time to create 1 rle with old method : 3.790855407714844e-05 time for calcul the mask position with numpy : 0.009567499160766602 nb_pixel_total : 56 time to create 1 rle with old method : 9.107589721679688e-05 time for calcul the mask position with numpy : 0.008671283721923828 nb_pixel_total : 1266 time to create 1 rle with old method : 0.0016646385192871094 time for calcul the mask position with numpy : 0.009261608123779297 nb_pixel_total : 141 time to create 1 rle with old method : 0.0002446174621582031 time for calcul the mask position with numpy : 0.008599281311035156 nb_pixel_total : 110 time to create 1 rle with old method : 0.00020766258239746094 time for calcul the mask position with numpy : 0.009119987487792969 nb_pixel_total : 994 time to create 1 rle with old method : 0.0013003349304199219 time for calcul the mask position with numpy : 0.010117530822753906 nb_pixel_total : 106798 time to create 1 rle with old method : 0.12649941444396973 time for calcul the mask position with numpy : 0.008646965026855469 nb_pixel_total : 175 time to create 1 rle with old method : 0.00024056434631347656 time for calcul the mask position with numpy : 0.008568286895751953 nb_pixel_total : 3594 time to create 1 rle with old method : 0.004157066345214844 time for calcul the mask position with numpy : 0.008641719818115234 nb_pixel_total : 1715 time to create 1 rle with old method : 0.0019724369049072266 time for calcul the mask position with numpy : 0.008666515350341797 nb_pixel_total : 130 time to create 1 rle with old method : 0.0002067089080810547 time for calcul the mask position with numpy : 0.008720159530639648 nb_pixel_total : 80 time to create 1 rle with old method : 0.0001544952392578125 time for calcul the mask position with numpy : 0.009359359741210938 nb_pixel_total : 202 time to create 1 rle with old method : 0.00026488304138183594 time for calcul the mask position with numpy : 0.008704185485839844 nb_pixel_total : 420 time to create 1 rle with old method : 0.0005533695220947266 time for calcul the mask position with numpy : 0.0087738037109375 nb_pixel_total : 1084 time to create 1 rle with old method : 0.0012624263763427734 time for calcul the mask position with numpy : 0.00851583480834961 nb_pixel_total : 1511 time to create 1 rle with old method : 0.0023238658905029297 time for calcul the mask position with numpy : 0.008637428283691406 nb_pixel_total : 199 time to create 1 rle with old method : 0.0002751350402832031 time for calcul the mask position with numpy : 0.008507251739501953 nb_pixel_total : 169 time to create 1 rle with old method : 0.00023102760314941406 time for calcul the mask position with numpy : 0.008635282516479492 nb_pixel_total : 878 time to create 1 rle with old method : 0.0011153221130371094 time for calcul the mask position with numpy : 0.00874638557434082 nb_pixel_total : 101 time to create 1 rle with old method : 0.00014066696166992188 time for calcul the mask position with numpy : 0.008867979049682617 nb_pixel_total : 1684 time to create 1 rle with old method : 0.002070188522338867 time for calcul the mask position with numpy : 0.008976221084594727 nb_pixel_total : 159 time to create 1 rle with old method : 0.00024700164794921875 time for calcul the mask position with numpy : 0.008523702621459961 nb_pixel_total : 1601 time to create 1 rle with old method : 0.0024385452270507812 time for calcul the mask position with numpy : 0.008594036102294922 nb_pixel_total : 366 time to create 1 rle with old method : 0.000453948974609375 time for calcul the mask position with numpy : 0.008434772491455078 nb_pixel_total : 158 time to create 1 rle with old method : 0.0002048015594482422 time for calcul the mask position with numpy : 0.008395671844482422 nb_pixel_total : 104 time to create 1 rle with old method : 0.0001327991485595703 time for calcul the mask position with numpy : 0.008545637130737305 nb_pixel_total : 291 time to create 1 rle with old method : 0.0003600120544433594 time for calcul the mask position with numpy : 0.008464574813842773 nb_pixel_total : 516 time to create 1 rle with old method : 0.0006530284881591797 time for calcul the mask position with numpy : 0.008708953857421875 nb_pixel_total : 124 time to create 1 rle with old method : 0.00021839141845703125 time for calcul the mask position with numpy : 0.009846687316894531 nb_pixel_total : 641 time to create 1 rle with old method : 0.0007872581481933594 time for calcul the mask position with numpy : 0.008558988571166992 nb_pixel_total : 896 time to create 1 rle with old method : 0.0010094642639160156 time for calcul the mask position with numpy : 0.01005864143371582 nb_pixel_total : 222 time to create 1 rle with old method : 0.00028586387634277344 time for calcul the mask position with numpy : 0.012326955795288086 nb_pixel_total : 364 time to create 1 rle with old method : 0.0004608631134033203 time for calcul the mask position with numpy : 0.008684158325195312 nb_pixel_total : 15 time to create 1 rle with old method : 4.649162292480469e-05 time for calcul the mask position with numpy : 0.011340618133544922 nb_pixel_total : 185 time to create 1 rle with old method : 0.0005292892456054688 time for calcul the mask position with numpy : 0.010890722274780273 nb_pixel_total : 355 time to create 1 rle with old method : 0.0006792545318603516 time for calcul the mask position with numpy : 0.011012077331542969 nb_pixel_total : 156 time to create 1 rle with old method : 0.0003306865692138672 time for calcul the mask position with numpy : 0.010656356811523438 nb_pixel_total : 589 time to create 1 rle with old method : 0.0010900497436523438 time for calcul the mask position with numpy : 0.010688066482543945 nb_pixel_total : 337 time to create 1 rle with old method : 0.0006504058837890625 time for calcul the mask position with numpy : 0.01134490966796875 nb_pixel_total : 104 time to create 1 rle with old method : 0.00020003318786621094 time for calcul the mask position with numpy : 0.01386260986328125 nb_pixel_total : 570 time to create 1 rle with old method : 0.0006921291351318359 time for calcul the mask position with numpy : 0.008371353149414062 nb_pixel_total : 382 time to create 1 rle with old method : 0.0004570484161376953 time for calcul the mask position with numpy : 0.008403301239013672 nb_pixel_total : 134 time to create 1 rle with old method : 0.00018715858459472656 time for calcul the mask position with numpy : 0.008455038070678711 nb_pixel_total : 167 time to create 1 rle with old method : 0.00020051002502441406 time for calcul the mask position with numpy : 0.00848531723022461 nb_pixel_total : 248 time to create 1 rle with old method : 0.0003056526184082031 time for calcul the mask position with numpy : 0.008487939834594727 nb_pixel_total : 200 time to create 1 rle with old method : 0.00023746490478515625 time for calcul the mask position with numpy : 0.009161233901977539 nb_pixel_total : 211 time to create 1 rle with old method : 0.0002753734588623047 create new chi : 1.782538890838623 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.010500431060791016 batch 1 Loaded 111 chid ids of type : 4230 Number RLEs to save : 9880 TO DO : save crop sub photo not yet done ! save time : 0.7649168968200684 nb_obj : 103 nb_hashtags : 5 time to prepare the origin masks : 2.3233108520507812 time for calcul the mask position with numpy : 0.04198861122131348 nb_pixel_total : 1874239 time to create 1 rle with new method : 0.07057499885559082 time for calcul the mask position with numpy : 0.012716293334960938 nb_pixel_total : 522 time to create 1 rle with old method : 0.0006647109985351562 time for calcul the mask position with numpy : 0.011698246002197266 nb_pixel_total : 342 time to create 1 rle with old method : 0.0004451274871826172 time for calcul the mask position with numpy : 0.011759757995605469 nb_pixel_total : 52 time to create 1 rle with old method : 0.00010514259338378906 time for calcul the mask position with numpy : 0.012104034423828125 nb_pixel_total : 183 time to create 1 rle with old method : 0.00023865699768066406 time for calcul the mask position with numpy : 0.012213468551635742 nb_pixel_total : 302 time to create 1 rle with old method : 0.00039386749267578125 time for calcul the mask position with numpy : 0.010395288467407227 nb_pixel_total : 1180 time to create 1 rle with old method : 0.0017447471618652344 time for calcul the mask position with numpy : 0.008810281753540039 nb_pixel_total : 1982 time to create 1 rle with old method : 0.0023071765899658203 time for calcul the mask position with numpy : 0.008306741714477539 nb_pixel_total : 426 time to create 1 rle with old method : 0.0004980564117431641 time for calcul the mask position with numpy : 0.007086992263793945 nb_pixel_total : 97 time to create 1 rle with old method : 0.000202178955078125 time for calcul the mask position with numpy : 0.0077817440032958984 nb_pixel_total : 1124 time to create 1 rle with old method : 0.0013427734375 time for calcul the mask position with numpy : 0.007563114166259766 nb_pixel_total : 873 time to create 1 rle with old method : 0.000995635986328125 time for calcul the mask position with numpy : 0.0075418949127197266 nb_pixel_total : 600 time to create 1 rle with old method : 0.000728607177734375 time for calcul the mask position with numpy : 0.0074503421783447266 nb_pixel_total : 120 time to create 1 rle with old method : 0.00017380714416503906 time for calcul the mask position with numpy : 0.0076563358306884766 nb_pixel_total : 46 time to create 1 rle with old method : 9.012222290039062e-05 time for calcul the mask position with numpy : 0.007216453552246094 nb_pixel_total : 12 time to create 1 rle with old method : 5.412101745605469e-05 time for calcul the mask position with numpy : 0.007988929748535156 nb_pixel_total : 357 time to create 1 rle with old method : 0.0004413127899169922 time for calcul the mask position with numpy : 0.007838010787963867 nb_pixel_total : 34 time to create 1 rle with old method : 8.416175842285156e-05 time for calcul the mask position with numpy : 0.008114099502563477 nb_pixel_total : 327 time to create 1 rle with old method : 0.0004298686981201172 time for calcul the mask position with numpy : 0.00812673568725586 nb_pixel_total : 223 time to create 1 rle with old method : 0.0002999305725097656 time for calcul the mask position with numpy : 0.008244752883911133 nb_pixel_total : 39 time to create 1 rle with old method : 0.00014257431030273438 time for calcul the mask position with numpy : 0.00789785385131836 nb_pixel_total : 4295 time to create 1 rle with old method : 0.004847526550292969 time for calcul the mask position with numpy : 0.007874488830566406 nb_pixel_total : 868 time to create 1 rle with old method : 0.0010280609130859375 time for calcul the mask position with numpy : 0.00785970687866211 nb_pixel_total : 1448 time to create 1 rle with old method : 0.0016734600067138672 time for calcul the mask position with numpy : 0.007578134536743164 nb_pixel_total : 79 time to create 1 rle with old method : 0.0001442432403564453 time for calcul the mask position with numpy : 0.007773876190185547 nb_pixel_total : 1574 time to create 1 rle with old method : 0.0018396377563476562 time for calcul the mask position with numpy : 0.007565975189208984 nb_pixel_total : 55 time to create 1 rle with old method : 9.989738464355469e-05 time for calcul the mask position with numpy : 0.0073223114013671875 nb_pixel_total : 279 time to create 1 rle with old method : 0.00041961669921875 time for calcul the mask position with numpy : 0.008423805236816406 nb_pixel_total : 225 time to create 1 rle with old method : 0.0002982616424560547 time for calcul the mask position with numpy : 0.008523702621459961 nb_pixel_total : 417 time to create 1 rle with old method : 0.0005342960357666016 time for calcul the mask position with numpy : 0.008350133895874023 nb_pixel_total : 4807 time to create 1 rle with old method : 0.005505561828613281 time for calcul the mask position with numpy : 0.007889032363891602 nb_pixel_total : 1254 time to create 1 rle with old method : 0.0014941692352294922 time for calcul the mask position with numpy : 0.008194446563720703 nb_pixel_total : 11035 time to create 1 rle with old method : 0.01258230209350586 time for calcul the mask position with numpy : 0.007623910903930664 nb_pixel_total : 1453 time to create 1 rle with old method : 0.0016574859619140625 time for calcul the mask position with numpy : 0.008721351623535156 nb_pixel_total : 743 time to create 1 rle with old method : 0.0009233951568603516 time for calcul the mask position with numpy : 0.008182287216186523 nb_pixel_total : 795 time to create 1 rle with old method : 0.0009603500366210938 time for calcul the mask position with numpy : 0.008437633514404297 nb_pixel_total : 1454 time to create 1 rle with old method : 0.0016913414001464844 time for calcul the mask position with numpy : 0.008053064346313477 nb_pixel_total : 348 time to create 1 rle with old method : 0.0004429817199707031 time for calcul the mask position with numpy : 0.010696172714233398 nb_pixel_total : 230 time to create 1 rle with old method : 0.00033855438232421875 time for calcul the mask position with numpy : 0.00874781608581543 nb_pixel_total : 432 time to create 1 rle with old method : 0.0005171298980712891 time for calcul the mask position with numpy : 0.008594512939453125 nb_pixel_total : 419 time to create 1 rle with old method : 0.0005240440368652344 time for calcul the mask position with numpy : 0.008840084075927734 nb_pixel_total : 1368 time to create 1 rle with old method : 0.0016677379608154297 time for calcul the mask position with numpy : 0.008548736572265625 nb_pixel_total : 235 time to create 1 rle with old method : 0.0003161430358886719 time for calcul the mask position with numpy : 0.008422374725341797 nb_pixel_total : 858 time to create 1 rle with old method : 0.0010957717895507812 time for calcul the mask position with numpy : 0.00858759880065918 nb_pixel_total : 76 time to create 1 rle with old method : 0.00012755393981933594 time for calcul the mask position with numpy : 0.008355855941772461 nb_pixel_total : 766 time to create 1 rle with old method : 0.0009493827819824219 time for calcul the mask position with numpy : 0.009122848510742188 nb_pixel_total : 834 time to create 1 rle with old method : 0.0010058879852294922 time for calcul the mask position with numpy : 0.008335351943969727 nb_pixel_total : 99 time to create 1 rle with old method : 0.00015044212341308594 time for calcul the mask position with numpy : 0.0076982975006103516 nb_pixel_total : 818 time to create 1 rle with old method : 0.0009264945983886719 time for calcul the mask position with numpy : 0.008190393447875977 nb_pixel_total : 56 time to create 1 rle with old method : 0.00010752677917480469 time for calcul the mask position with numpy : 0.008275270462036133 nb_pixel_total : 123 time to create 1 rle with old method : 0.00018286705017089844 time for calcul the mask position with numpy : 0.008161783218383789 nb_pixel_total : 1242 time to create 1 rle with old method : 0.0014858245849609375 time for calcul the mask position with numpy : 0.008195877075195312 nb_pixel_total : 734 time to create 1 rle with old method : 0.0008952617645263672 time for calcul the mask position with numpy : 0.008239507675170898 nb_pixel_total : 97 time to create 1 rle with old method : 0.00018310546875 time for calcul the mask position with numpy : 0.008343935012817383 nb_pixel_total : 1426 time to create 1 rle with old method : 0.0017015933990478516 time for calcul the mask position with numpy : 0.007748603820800781 nb_pixel_total : 282 time to create 1 rle with old method : 0.00037550926208496094 time for calcul the mask position with numpy : 0.008202314376831055 nb_pixel_total : 9036 time to create 1 rle with old method : 0.010498762130737305 time for calcul the mask position with numpy : 0.0075337886810302734 nb_pixel_total : 1104 time to create 1 rle with old method : 0.0012521743774414062 time for calcul the mask position with numpy : 0.008324146270751953 nb_pixel_total : 1357 time to create 1 rle with old method : 0.0017418861389160156 time for calcul the mask position with numpy : 0.008105993270874023 nb_pixel_total : 1814 time to create 1 rle with old method : 0.002150297164916992 time for calcul the mask position with numpy : 0.007916688919067383 nb_pixel_total : 2315 time to create 1 rle with old method : 0.002713918685913086 time for calcul the mask position with numpy : 0.007409095764160156 nb_pixel_total : 956 time to create 1 rle with old method : 0.0011987686157226562 time for calcul the mask position with numpy : 0.007661342620849609 nb_pixel_total : 872 time to create 1 rle with old method : 0.001050710678100586 time for calcul the mask position with numpy : 0.007522106170654297 nb_pixel_total : 69 time to create 1 rle with old method : 0.00011730194091796875 time for calcul the mask position with numpy : 0.007415056228637695 nb_pixel_total : 1352 time to create 1 rle with old method : 0.0017037391662597656 time for calcul the mask position with numpy : 0.007166147232055664 nb_pixel_total : 2635 time to create 1 rle with old method : 0.002982616424560547 time for calcul the mask position with numpy : 0.007250547409057617 nb_pixel_total : 741 time to create 1 rle with old method : 0.0009984970092773438 time for calcul the mask position with numpy : 0.009895801544189453 nb_pixel_total : 62 time to create 1 rle with old method : 0.00013208389282226562 time for calcul the mask position with numpy : 0.008183479309082031 nb_pixel_total : 1536 time to create 1 rle with old method : 0.0018453598022460938 time for calcul the mask position with numpy : 0.006226539611816406 nb_pixel_total : 72 time to create 1 rle with old method : 0.00010943412780761719 time for calcul the mask position with numpy : 0.0062236785888671875 nb_pixel_total : 1376 time to create 1 rle with old method : 0.0016465187072753906 time for calcul the mask position with numpy : 0.008148670196533203 nb_pixel_total : 965 time to create 1 rle with old method : 0.0011494159698486328 time for calcul the mask position with numpy : 0.006378889083862305 nb_pixel_total : 74 time to create 1 rle with old method : 0.00014829635620117188 time for calcul the mask position with numpy : 0.0078029632568359375 nb_pixel_total : 106818 time to create 1 rle with old method : 0.12996816635131836 time for calcul the mask position with numpy : 0.008085250854492188 nb_pixel_total : 3682 time to create 1 rle with old method : 0.005541801452636719 time for calcul the mask position with numpy : 0.009582757949829102 nb_pixel_total : 1591 time to create 1 rle with old method : 0.004313230514526367 time for calcul the mask position with numpy : 0.008228063583374023 nb_pixel_total : 274 time to create 1 rle with old method : 0.0004677772521972656 time for calcul the mask position with numpy : 0.009449243545532227 nb_pixel_total : 87 time to create 1 rle with old method : 0.0001373291015625 time for calcul the mask position with numpy : 0.00785207748413086 nb_pixel_total : 148 time to create 1 rle with old method : 0.00023055076599121094 time for calcul the mask position with numpy : 0.007798910140991211 nb_pixel_total : 413 time to create 1 rle with old method : 0.0005667209625244141 time for calcul the mask position with numpy : 0.007619380950927734 nb_pixel_total : 1618 time to create 1 rle with old method : 0.0022983551025390625 time for calcul the mask position with numpy : 0.007616519927978516 nb_pixel_total : 827 time to create 1 rle with old method : 0.001190185546875 time for calcul the mask position with numpy : 0.007688999176025391 nb_pixel_total : 162 time to create 1 rle with old method : 0.0002579689025878906 time for calcul the mask position with numpy : 0.0072479248046875 nb_pixel_total : 807 time to create 1 rle with old method : 0.0011398792266845703 time for calcul the mask position with numpy : 0.007302761077880859 nb_pixel_total : 130 time to create 1 rle with old method : 0.0002288818359375 time for calcul the mask position with numpy : 0.008970022201538086 nb_pixel_total : 1716 time to create 1 rle with old method : 0.0022368431091308594 time for calcul the mask position with numpy : 0.007305145263671875 nb_pixel_total : 396 time to create 1 rle with old method : 0.0005784034729003906 time for calcul the mask position with numpy : 0.007582664489746094 nb_pixel_total : 114 time to create 1 rle with old method : 0.00020194053649902344 time for calcul the mask position with numpy : 0.007515668869018555 nb_pixel_total : 116 time to create 1 rle with old method : 0.00025153160095214844 time for calcul the mask position with numpy : 0.007027149200439453 nb_pixel_total : 127 time to create 1 rle with old method : 0.00020503997802734375 time for calcul the mask position with numpy : 0.007704257965087891 nb_pixel_total : 649 time to create 1 rle with old method : 0.0009353160858154297 time for calcul the mask position with numpy : 0.0076711177825927734 nb_pixel_total : 194 time to create 1 rle with old method : 0.0002961158752441406 time for calcul the mask position with numpy : 0.007879257202148438 nb_pixel_total : 17 time to create 1 rle with old method : 5.030632019042969e-05 time for calcul the mask position with numpy : 0.009427785873413086 nb_pixel_total : 70 time to create 1 rle with old method : 0.0008695125579833984 time for calcul the mask position with numpy : 0.007409572601318359 nb_pixel_total : 352 time to create 1 rle with old method : 0.0008053779602050781 time for calcul the mask position with numpy : 0.008318662643432617 nb_pixel_total : 161 time to create 1 rle with old method : 0.00040268898010253906 time for calcul the mask position with numpy : 0.008880376815795898 nb_pixel_total : 142 time to create 1 rle with old method : 0.0002541542053222656 time for calcul the mask position with numpy : 0.008283615112304688 nb_pixel_total : 475 time to create 1 rle with old method : 0.0007498264312744141 time for calcul the mask position with numpy : 0.007401943206787109 nb_pixel_total : 356 time to create 1 rle with old method : 0.0004515647888183594 time for calcul the mask position with numpy : 0.0077626705169677734 nb_pixel_total : 99 time to create 1 rle with old method : 0.00017881393432617188 time for calcul the mask position with numpy : 0.006869792938232422 nb_pixel_total : 506 time to create 1 rle with old method : 0.0006403923034667969 time for calcul the mask position with numpy : 0.007212162017822266 nb_pixel_total : 569 time to create 1 rle with old method : 0.0008816719055175781 time for calcul the mask position with numpy : 0.0075566768646240234 nb_pixel_total : 192 time to create 1 rle with old method : 0.0003123283386230469 time for calcul the mask position with numpy : 0.007532596588134766 nb_pixel_total : 152 time to create 1 rle with old method : 0.0002474784851074219 create new chi : 1.2094159126281738 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.0014979839324951172 batch 1 Loaded 104 chid ids of type : 4230 Number RLEs to save : 9810 TO DO : save crop sub photo not yet done ! save time : 0.7133381366729736 map_output_result : {1337616822: (0.0, 'Should be the crop_list due to order', 0.0), 1337616763: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1337616822, 1337616763] Looping around the photos to save general results len do output : 2 /1337616822.Didn't retrieve data . /1337616763.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2589744') ('4234', '20578763', '1337616822', None, None, None, None, None, '2589744') ('4234', None, None, None, None, None, None, None, '2589744') ('4234', '20578763', '1337616763', None, None, None, None, None, '2589744') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.013963937759399414 save_final save missing photos in datou_result : time spend for datou_step_exec : 8.395369291305542 time spend to save output : 0.014183759689331055 total time spend for step 4 : 8.409553050994873 step5:crop_condition Sat Feb 15 00:21:04 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4230 Loading chi in step crop for list_pids : 2 ! batch 1 Loaded 215 chid ids of type : 4230 begin to crop the class : papier param for this class : {'min_score': 0.6} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 78 About to insert : list_path_to_insert length 78 new photo from crops ! About to upload 78 photos upload in portfolio : 4869462 init cache_photo without model_param we have 78 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739575266_2958243 we have uploaded 78 photos in the portfolio 4869462 time of upload the photos Elapsed time : 17.582542419433594 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.6} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 25 About to insert : list_path_to_insert length 25 new photo from crops ! About to upload 25 photos upload in portfolio : 4869462 init cache_photo without model_param we have 25 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739575284_2958243 we have uploaded 25 photos in the portfolio 4869462 time of upload the photos Elapsed time : 6.542973518371582 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.6} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.6} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 10 About to insert : list_path_to_insert length 10 new photo from crops ! About to upload 10 photos upload in portfolio : 4869462 init cache_photo without model_param we have 10 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739575291_2958243 we have uploaded 10 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.8474280834198 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.6} filtre for class : autre hashtag_id of this class : 494826614 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 37 About to insert : list_path_to_insert length 37 new photo from crops ! About to upload 37 photos upload in portfolio : 4869462 init cache_photo without model_param we have 37 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739575294_2958243 we have uploaded 37 photos in the portfolio 4869462 time of upload the photos Elapsed time : 12.193665504455566 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.6} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.6} filtre for class : pet_fonce hashtag_id of this class : 2107755900 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739575307_2958243 we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.0003271102905273 we have finished the crop for the class : pet_fonce delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1337616822, 1337616763] Looping around the photos to save general results len do output : 151 /1337751889Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751891Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751893Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751895Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751897Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751899Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751901Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751903Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751905Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751907Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751909Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751911Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751913Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751915Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751917Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751919Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751921Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751923Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751925Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751927Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751929Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751931Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751933Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751935Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751937Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751939Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751941Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751943Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751945Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751947Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751949Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751951Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751953Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751955Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751957Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751959Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751961Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751963Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751965Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751967Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751969Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751971Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751973Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751975Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751977Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751979Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751981Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751983Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751985Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751987Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751989Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751991Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751993Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751995Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751997Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337751999Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752001Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752004Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752006Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752008Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752010Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752012Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752014Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752016Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752018Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752020Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752022Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752024Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752026Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752028Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752030Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752032Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752034Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752036Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752038Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752040Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752042Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752044Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752154Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752156Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752158Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752160Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752162Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752164Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752166Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752168Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752170Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752172Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752174Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752176Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752178Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752180Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752182Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752184Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752186Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752188Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752190Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752192Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752194Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752196Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752198Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752200Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752202Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752255Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752257Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752259Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752261Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752263Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752265Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752267Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752269Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752271Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752273Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752323Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752324Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752325Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752327Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752328Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752329Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752330Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752331Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752332Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752333Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752334Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752335Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752336Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752337Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752338Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752339Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752340Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752341Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752342Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752343Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752344Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752345Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752346Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752347Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752348Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752349Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752350Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752351Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752352Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752353Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752354Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752355Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752356Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752357Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752358Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752359Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752360Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337752361Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2589744') ('4234', '20578763', '1337616822', None, None, None, None, None, '2589744') ('4234', None, None, None, None, None, None, None, '2589744') ('4234', '20578763', '1337616763', None, None, None, None, None, '2589744') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 455 time used for this insertion : 1.7467608451843262 save_final save missing photos in datou_result : time spend for datou_step_exec : 43.12364888191223 time spend to save output : 1.7500436305999756 total time spend for step 5 : 44.87369251251221 step6:thcl Sat Feb 15 00:21:49 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! we are using the classfication for only one thcl 3237 time to import caffe and check if the image exist : 0.00529026985168457 time to convert the images to numpy array : 0.013474225997924805 time to import caffe and check if the image exist : 0.012307405471801758 time to convert the images to numpy array : 0.03113555908203125 time to import caffe and check if the image exist : 0.011829853057861328 time to convert the images to numpy array : 0.04385232925415039 time to import caffe and check if the image exist : 0.01741337776184082 time to convert the images to numpy array : 0.043100595474243164 time to import caffe and check if the image exist : 0.0204312801361084 time to convert the images to numpy array : 0.044089555740356445 time to import caffe and check if the image exist : 0.02736520767211914 time to convert the images to numpy array : 0.03794670104980469 time to import caffe and check if the image exist : 0.029897212982177734 time to convert the images to numpy array : 0.03989076614379883 time to import caffe and check if the image exist : 0.015697956085205078 time to convert the images to numpy array : 0.055037736892700195 time to import caffe and check if the image exist : 0.024957895278930664 time to convert the images to numpy array : 0.046282291412353516 time to import caffe and check if the image exist : 0.026169300079345703 time to convert the images to numpy array : 0.04535627365112305 total time to convert the images to numpy array : 0.4087412357330322 list photo_ids error: [] list photo_ids correct : [1337752355, 1337752356, 1337752357, 1337752358, 1337752359, 1337752360, 1337752361, 1337751921, 1337751923, 1337751925, 1337751927, 1337751929, 1337751931, 1337751933, 1337751935, 1337751937, 1337751939, 1337751941, 1337751943, 1337751945, 1337751947, 1337751949, 1337751951, 1337751889, 1337751891, 1337751893, 1337751895, 1337751897, 1337751899, 1337751901, 1337751903, 1337751905, 1337751907, 1337751909, 1337751911, 1337751913, 1337751915, 1337751917, 1337751919, 1337752339, 1337752340, 1337752341, 1337752342, 1337752343, 1337752344, 1337752345, 1337752346, 1337752347, 1337752348, 1337752349, 1337752350, 1337752351, 1337752352, 1337752353, 1337752354, 1337752018, 1337752020, 1337752022, 1337752024, 1337752026, 1337752028, 1337752030, 1337752032, 1337752034, 1337752036, 1337752038, 1337752040, 1337752042, 1337752044, 1337752154, 1337752156, 1337751985, 1337751987, 1337751989, 1337751991, 1337751993, 1337751995, 1337751997, 1337751999, 1337752001, 1337752004, 1337752006, 1337752008, 1337752010, 1337752012, 1337752014, 1337752016, 1337752273, 1337752323, 1337752324, 1337752325, 1337752327, 1337752328, 1337752329, 1337752330, 1337752331, 1337752332, 1337752333, 1337752334, 1337752335, 1337752336, 1337752337, 1337752338, 1337752190, 1337752192, 1337752194, 1337752196, 1337752198, 1337752200, 1337752202, 1337752255, 1337752257, 1337752259, 1337752261, 1337752263, 1337752265, 1337752267, 1337752269, 1337752271, 1337752158, 1337752160, 1337752162, 1337752164, 1337752166, 1337752168, 1337752170, 1337752172, 1337752174, 1337752176, 1337752178, 1337752180, 1337752182, 1337752184, 1337752186, 1337752188, 1337751953, 1337751955, 1337751957, 1337751959, 1337751961, 1337751963, 1337751965, 1337751967, 1337751969, 1337751971, 1337751973, 1337751975, 1337751977, 1337751979, 1337751981, 1337751983] number of photos to traite : 151 try to delete the photos incorrect in DB tagging for thcl : 3237 To do loadFromThcl(), then load ParamDescType : thcl3237 thcls : [{'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'}] thcl {'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'} Update svm_hashtag_type_desc : 5561 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) To loadFromThcl() : net_5561 begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 2500 l 3632 free memory gpu now : 567 wait 20 seconds l 3637 free memory gpu now : 567 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_rubbia_refus_2500 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_rubbia_refus_2500 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_rubbia_refus_2500 /data/models_weight/learn_rubbia_refus_2500/caffemodel size_local : 94358479 size in s3 : 94358479 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:16 caffemodel already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt size_local : 32544 size in s3 : 32544 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:15 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:09:52 mean.npy already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/synset_words.txt size_local : 334 size in s3 : 334 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:10:06 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt caffemodel_filename : /data/models_weight/learn_rubbia_refus_2500/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 5472 max_wait_temp : 1 max_wait : 0 dict_keys(['prob', 'pool5']) time used to do the prepocess of the images : 0.7441694736480713 time used to do the prediction : 0.6432173252105713 save descriptor for thcl : 3237 time to traite the descriptors : 0.9160256385803223 Catched exception ! Connect or reconnect ! storage_type for insertDescriptorsMulti : 3 To insert : 1337752355 To insert : 1337752356 To insert : 1337752357 To insert : 1337752358 To insert : 1337752359 To insert : 1337752360 To insert : 1337752361 To insert : 1337751921 To insert : 1337751923 To insert : 1337751925 To insert : 1337751927 To insert : 1337751929 To insert : 1337751931 To insert : 1337751933 To insert : 1337751935 To insert : 1337751937 To insert : 1337751939 To insert : 1337751941 To insert : 1337751943 To insert : 1337751945 To insert : 1337751947 To insert : 1337751949 To insert : 1337751951 To insert : 1337751889 To insert : 1337751891 To insert : 1337751893 To insert : 1337751895 To insert : 1337751897 To insert : 1337751899 To insert : 1337751901 To insert : 1337751903 To insert : 1337751905 To insert : 1337751907 To insert : 1337751909 To insert : 1337751911 To insert : 1337751913 To insert : 1337751915 To insert : 1337751917 To insert : 1337751919 To insert : 1337752339 To insert : 1337752340 To insert : 1337752341 To insert : 1337752342 To insert : 1337752343 To insert : 1337752344 To insert : 1337752345 To insert : 1337752346 To insert : 1337752347 To insert : 1337752348 To insert : 1337752349 To insert : 1337752350 To insert : 1337752351 To insert : 1337752352 To insert : 1337752353 To insert : 1337752354 To insert : 1337752018 To insert : 1337752020 To insert : 1337752022 To insert : 1337752024 To insert : 1337752026 To insert : 1337752028 To insert : 1337752030 To insert : 1337752032 To insert : 1337752034 To insert : 1337752036 To insert : 1337752038 To insert : 1337752040 To insert : 1337752042 To insert : 1337752044 To insert : 1337752154 To insert : 1337752156 To insert : 1337751985 To insert : 1337751987 To insert : 1337751989 To insert : 1337751991 To insert : 1337751993 To insert : 1337751995 To insert : 1337751997 To insert : 1337751999 To insert : 1337752001 To insert : 1337752004 To insert : 1337752006 To insert : 1337752008 To insert : 1337752010 To insert : 1337752012 To insert : 1337752014 To insert : 1337752016 To insert : 1337752273 To insert : 1337752323 To insert : 1337752324 To insert : 1337752325 To insert : 1337752327 To insert : 1337752328 To insert : 1337752329 To insert : 1337752330 To insert : 1337752331 To insert : 1337752332 To insert : 1337752333 To insert : 1337752334 To insert : 1337752335 To insert : 1337752336 To insert : 1337752337 To insert : 1337752338 To insert : 1337752190 To insert : 1337752192 To insert : 1337752194 To insert : 1337752196 To insert : 1337752198 To insert : 1337752200 To insert : 1337752202 To insert : 1337752255 To insert : 1337752257 To insert : 1337752259 To insert : 1337752261 To insert : 1337752263 To insert : 1337752265 To insert : 1337752267 To insert : 1337752269 To insert : 1337752271 To insert : 1337752158 To insert : 1337752160 To insert : 1337752162 To insert : 1337752164 To insert : 1337752166 To insert : 1337752168 To insert : 1337752170 To insert : 1337752172 To insert : 1337752174 To insert : 1337752176 To insert : 1337752178 To insert : 1337752180 To insert : 1337752182 To insert : 1337752184 To insert : 1337752186 To insert : 1337752188 To insert : 1337751953 To insert : 1337751955 To insert : 1337751957 To insert : 1337751959 To insert : 1337751961 To insert : 1337751963 To insert : 1337751965 To insert : 1337751967 To insert : 1337751969 To insert : 1337751971 To insert : 1337751973 To insert : 1337751975 To insert : 1337751977 To insert : 1337751979 To insert : 1337751981 To insert : 1337751983 time to insert the descriptors : 30.149973154067993 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos SAVE THCL : begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 228 time used for this insertion : 0.021651744842529297 save missing photos in datou_result : time spend for datou_step_exec : 57.375901222229004 time spend to save output : 0.06896591186523438 total time spend for step 6 : 57.44486713409424 step7:ventilate_hashtags_in_portfolio Sat Feb 15 00:22:47 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 20578763 get user id for portfolio 20578763 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20578763 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('PET_clair','papier','mal_croppe','Film_plastique','pet_clair','PET_fonce','carton','pehd','pet_fonce','Tetrapak','autre','metal','environnement','refus','PEHD','Papier','flou','Carton')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20578763 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('PET_clair','papier','mal_croppe','Film_plastique','pet_clair','PET_fonce','carton','pehd','pet_fonce','Tetrapak','autre','metal','environnement','refus','PEHD','Papier','flou','Carton')) AND mptpi.`min_score`=0.6 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20578763 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('PET_clair','papier','mal_croppe','Film_plastique','pet_clair','PET_fonce','carton','pehd','pet_fonce','Tetrapak','autre','metal','environnement','refus','PEHD','Papier','flou','Carton')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20578763 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('PET_clair','papier','mal_croppe','Film_plastique','pet_clair','PET_fonce','carton','pehd','pet_fonce','Tetrapak','autre','metal','environnement','refus','PEHD','Papier','flou','Carton')) AND mptpi.`min_score`=0.6 To do lien utilise dans velours : https://www.fotonower.com/velours/20579009,20579020,20579007,20579008,20579013,20579022,20579019,20579014,20579015,20579016,20579017,20579018,20579021?tags=pet_clair,papier,mal_croppe,film_plastique,pet_fonce,carton,pehd,tetrapak,autre,metal,environnement,refus,flou&datou_id_consolidate=4235&port_consolidate=20578763 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1337616822, 1337616763] Looping around the photos to save general results len do output : 1 /20578763. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2589744') ('4234', '20578763', '1337616822', None, None, None, None, None, '2589744') ('4234', None, None, None, None, None, None, None, '2589744') ('4234', '20578763', '1337616763', None, None, None, None, None, '2589744') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.016102075576782227 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.3657803535461426 time spend to save output : 0.01633739471435547 total time spend for step 7 : 2.382117748260498 step8:final Sat Feb 15 00:22:49 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1337616822: ('0.12385820099779768',), 1337616763: ('0.12385820099779768',)} new output for save of step final : {1337616822: ('0.12385820099779768',), 1337616763: ('0.12385820099779768',)} [1337616822, 1337616763] Looping around the photos to save general results len do output : 2 /1337616822.Didn't retrieve data . /1337616763.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2589744') ('4234', '20578763', '1337616822', None, None, None, None, None, '2589744') ('4234', None, None, None, None, None, None, None, '2589744') ('4234', '20578763', '1337616763', None, None, None, None, None, '2589744') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.01184844970703125 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.12159585952758789 time spend to save output : 0.01209712028503418 total time spend for step 8 : 0.13369297981262207 step9:velours_tree Sat Feb 15 00:22:49 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure list_portfolios : 20579009,20579020,20579007,20579008,20579013,20579022,20579019,20579014,20579015,20579016,20579017,20579018,20579021 photo desc type : 5561 - Retrieving photos to tag... query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20579009 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20579020 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20579007 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20579008 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20579013 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20579022 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20579019 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20579014 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20579015 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20579016 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20579017 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20579018 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20579021 ORDER BY ph.size desc - Loading descriptors... Size : 2048 len(descriptors) : 151 Compute structured hierarchical clustering... ward : AgglomerativeClustering(n_clusters=6) ward.labels_ : [1 1 1 4 1 0 3 4 4 3 0 3 0 4 1 1 3 1 1 0 0 1 0 1 1 0 0 1 0 3 0 1 1 1 0 4 3 5 0 4 1 0 4 3 0 3 4 4 3 3 1 0 5 4 1 1 0 1 1 1 1 0 1 1 1 0 1 1 1 1 0 4 0 3 5 0 1 0 2 0 0 1 5 3 3 5 0 0 1 1 2 5 2 0 1 1 1 1 1 3 3 5 4 0 5 1 0 4 4 5 1 1 0 1 0 1 2 1 0 4 4 0 1 2 3 1 1 5 4 0 1 4 1 1 1 1 0 1 0 1 1 2 5 0 0 1 4 1 5 4 0] Elapsed time: 0.028119802474975586 graph_id used : 78723 - Beta version, working pretty good on 11-5-16 ! https://www.fotonower.com/velours/20579009,20579020,20579007,20579008,20579013,20579022,20579019,20579014,20579015,20579016,20579017,20579018,20579021?tags=pet_clair,papier,mal_croppe,film_plastique,pet_fonce,carton,pehd,tetrapak,autre,metal,environnement,refus,flou&datou_id_consolidate=4235&port_consolidate=20578763&tree_id=78723 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : velours_tree we use saveGeneral [1337616822, 1337616763] Looping around the photos to save general results len do output : 1 /20578763Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2589744') ('4234', '20578763', '1337616822', None, None, None, None, None, '2589744') ('4234', None, None, None, None, None, None, None, '2589744') ('4234', '20578763', '1337616763', None, None, None, None, None, '2589744') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.019225120544433594 save_final save missing photos in datou_result : time spend for datou_step_exec : 10.416565179824829 time spend to save output : 0.01951766014099121 total time spend for step 9 : 10.43608283996582 step10:send_mail_cod Sat Feb 15 00:23:00 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Qualipapia_P20578763_15-02-2025_00_23_00.pdf 20578991 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette205789911739575380 20579002 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette205790021739575380 20578989 imagette205789891739575381 20578990 imagette205789901739575381 20578995 change filename to text .imagette205789951739575381 20579004 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette205790041739575381 20579001 imagette205790011739575382 20578996 imagette205789961739575382 20578997 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette205789971739575382 20578998 imagette205789981739575383 20579000 imagette205790001739575383 20579003 imagette205790031739575383 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20578763 and hashtag_type = 4230 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20579009,20579020,20579007,20579008,20579013,20579022,20579019,20579014,20579015,20579016,20579017,20579018,20579021?tags=pet_clair,papier,mal_croppe,film_plastique,pet_fonce,carton,pehd,tetrapak,autre,metal,environnement,refus,flou&datou_id_consolidate=4235&port_consolidate=20578763&tree_id=78723 your option no_mail is active, we will not send the real mail to your client args[1337616822] : ((1337616822, -7.959349608786445, 492609224), (1337616822, -0.3716630697235064, 496442774), '0.12385820099779768') apple ((1337616822, -7.959349608786445, 492609224), (1337616822, -0.3716630697235064, 496442774), '0.12385820099779768') We are sending mail with results at cod@fotonower.com args[1337616763] : ((1337616763, -7.959972801904531, 492609224), (1337616763, -0.35480994244737557, 496442774), '0.12385820099779768') apple ((1337616763, -7.959972801904531, 492609224), (1337616763, -0.35480994244737557, 496442774), '0.12385820099779768') We are sending mail with results at cod@fotonower.com refus_total : 0.12385820099779768 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20578763 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20578763_15-02-2025_00_23_00.pdf results_Qualipapia_P20578763_15-02-2025_00_23_00.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20578763_15-02-2025_00_23_00.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4234','20578763','results_Qualipapia_P20578763_15-02-2025_00_23_00.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20578763_15-02-2025_00_23_00.pdf','pdf','','0.15','0.12385820099779768') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1337616822, 1337616763] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2589744') ('4234', '20578763', '1337616822', None, None, None, None, None, '2589744') ('4234', None, None, None, None, None, None, None, '2589744') ('4234', '20578763', '1337616763', None, None, None, None, None, '2589744') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 2 time used for this insertion : 0.014705896377563477 save_final save missing photos in datou_result : time spend for datou_step_exec : 3.972144365310669 time spend to save output : 0.014878988265991211 total time spend for step 10 : 3.98702335357666 step11:split_time_score Sat Feb 15 00:23:04 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 3442, 'mtr_user_id': 31, 'name': 'classifieur_2camions_valcor_021122_v1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'deux_camions,camion_droite,camion_gauche,pas_de_camion', 'svm_portfolios_learning': '7659379,7659034,7657685,7657114', 'photo_hashtag_type': 4458, 'photo_desc_type': 5723, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107760533,2107760534,2107760535,2107760536'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('11', 2),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 14022025 20578763 Nombre de photos uploadées : 2 / 23040 (0%) 14022025 20578763 Nombre de photos taguées (types de déchets): 2 / 2 (100%) 14022025 20578763 Nombre de photos taguées (volume) : 0 / 2 (0%) elapsed_time : load_data_split_time_score 1.9073486328125e-06 elapsed_time : order_list_meta_photo_and_scores 5.4836273193359375e-06 ?? elapsed_time : fill_and_build_computed_from_old_data 0.00022935867309570312 elapsed_time : insert_dashboard_record_day_entry 0.028181076049804688 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.14080265003567735 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20557267_14-02-2025_13_16_58.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20557267 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20557267 AND mptpi.`type`=4230 To do Qualite : 0.16283505865355488 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20557268_14-02-2025_13_06_07.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20557268 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20557268 AND mptpi.`type`=4230 To do Qualite : 0.12385820099779768 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20578763_15-02-2025_00_23_00.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20578763 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20578763 AND mptpi.`type`=4230 To do Qualite : 0.17080943635979126 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20561437_14-02-2025_16_47_13.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20561437 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20561437 AND mptpi.`type`=4230 To do Qualite : 0.15705737530364391 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20561438_14-02-2025_16_22_18.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20561438 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20561438 AND mptpi.`type`=4230 To do Qualite : 0.24123338678301143 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20561439_14-02-2025_16_36_29.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20561439 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20561439 AND mptpi.`type`=4230 To do Qualite : 0.2278315065084019 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20561440_14-02-2025_15_49_05.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20561440 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20561440 AND mptpi.`type`=4230 To do Qualite : 0.21292784892651612 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20577017_14-02-2025_22_03_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20577017 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20577017 AND mptpi.`type`=4230 To do Qualite : 0.21228317775297423 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20565004_14-02-2025_17_33_07.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20565004 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20565004 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20565005 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20565006 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20565007 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20568568 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20568570 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20568571 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20568572 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20568575 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20568587 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20568601 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20568602 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567453 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567454 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567455 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567456 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567457 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567458 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567459 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567460 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567461 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567462 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567463 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567464 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567465 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567466 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567467 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567468 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567469 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567470 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567471 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567472 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567473 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567474 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567475 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567476 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567477 order by id desc limit 1 Qualite : 0.23699891677009535 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20567478_14-02-2025_18_34_55.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567478 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20567478 AND mptpi.`type`=4230 To do Qualite : 0.2659324169831625 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20567479_14-02-2025_18_20_05.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567479 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20567479 AND mptpi.`type`=4230 To do Qualite : 0.23264297907652698 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20567480_14-02-2025_18_23_12.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20567480 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20567480 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20568604 order by id desc limit 1 Qualite : 0.22023519577147693 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20568605_14-02-2025_23_48_20.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20568605 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20568605 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20568606 order by id desc limit 1 Qualite : 0.25440762275141415 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20568607_14-02-2025_20_23_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20568607 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20568607 AND mptpi.`type`=4230 To do Qualite : 0.23908227128633877 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20568608_14-02-2025_20_21_27.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20568608 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20568608 AND mptpi.`type`=4230 To do Qualite : 0.25379507447780547 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20568609_14-02-2025_18_47_29.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20568609 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20568609 AND mptpi.`type`=4230 To do Qualite : 0.255508684248806 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20568610_14-02-2025_19_35_29.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20568610 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20568610 AND mptpi.`type`=4230 To do Qualite : 0.24818222150458366 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20568611_14-02-2025_20_34_57.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20568611 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20568611 AND mptpi.`type`=4230 To do Qualite : 0.2428419785256863 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20568612_14-02-2025_18_37_43.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20568612 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20568612 AND mptpi.`type`=4230 To do Qualite : 0.2477337637088999 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20571465_14-02-2025_19_47_58.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20571465 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20571465 AND mptpi.`type`=4230 To do Qualite : 0.23064257970638882 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20576591_14-02-2025_22_45_27.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20576591 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20576591 AND mptpi.`type`=4230 To do Qualite : 0.21982307840160847 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20576592_14-02-2025_23_29_26.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20576592 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20576592 AND mptpi.`type`=4230 To do Qualite : 0.21935584026591903 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20575968_14-02-2025_23_32_19.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20575968 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20575968 AND mptpi.`type`=4230 To do Qualite : 0.20981345687102668 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20575969_14-02-2025_21_19_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20575969 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20575969 AND mptpi.`type`=4230 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'14022025': {'nb_upload': 2, 'nb_taggue_class': 2, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1337616822, 1337616763] Looping around the photos to save general results len do output : 1 /20578763Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2589744') ('4234', '20578763', '1337616822', None, None, None, None, None, '2589744') ('4234', None, None, None, None, None, None, None, '2589744') ('4234', '20578763', '1337616763', None, None, None, None, None, '2589744') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.01682305335998535 save_final save missing photos in datou_result : time spend for datou_step_exec : 31.114050149917603 time spend to save output : 0.017099380493164062 total time spend for step 11 : 31.131149530410767 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 1 len(input) + len(total_photo_id_missing) : 2 set_done_treatment 32.78user 23.48system 3:10.36elapsed 29%CPU (0avgtext+0avgdata 2454128maxresident)k 382440inputs+11640outputs (1765major+1991089minor)pagefaults 0swaps