python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 4234 -l 5 ' -s datou_current_4234 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1861816 load datou : 4234 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load THCL from format json or kwargs add thcl : 3237 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig add pdt : 5561 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 we have a portfolio with more photos than limit : 45>5 please execute split_portfolio.py -i 20587832 -l 5 size over we load limit photo not treated list_input_json: [] Current got : datou_id : 4234, datou_cur_ids : ['2591081'] with mtr_portfolio_ids : ['20587832'] and first list_photo_ids : [] new path : /proc/1861816/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, brightness, blur_detection, rle_unique_nms_with_priority, crop_condition, thcl, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 3 ; length of list_pids : 3 ; length of list_args : 3 time to download the photos : 0.4028284549713135 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Sat Feb 15 19:20:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10770 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-15 19:20:31.398250: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-15 19:20:31.423043: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-15 19:20:31.424532: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f6514000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-15 19:20:31.424586: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-15 19:20:31.427331: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-15 19:20:31.537657: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x235c41f0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-15 19:20:31.537703: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-15 19:20:31.539237: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-15 19:20:31.539653: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-15 19:20:31.542775: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-15 19:20:31.545769: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-15 19:20:31.546270: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-15 19:20:31.548896: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-15 19:20:31.549896: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-15 19:20:31.554265: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-15 19:20:31.555733: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-15 19:20:31.555808: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-15 19:20:31.556547: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-15 19:20:31.556562: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-15 19:20:31.556571: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-15 19:20:31.557855: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9837 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-15 19:20:31.809565: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-15 19:20:31.809715: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-15 19:20:31.810168: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-15 19:20:31.810204: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-15 19:20:31.810234: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-15 19:20:31.810265: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-15 19:20:31.810294: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-15 19:20:31.810324: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-15 19:20:31.812192: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-15 19:20:31.813382: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-15 19:20:31.813413: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-15 19:20:31.813428: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-15 19:20:31.813442: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-15 19:20:31.813456: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-15 19:20:31.813471: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-15 19:20:31.813484: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-15 19:20:31.813499: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-15 19:20:31.814665: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-15 19:20:31.814698: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-15 19:20:31.814706: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-15 19:20:31.814713: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-15 19:20:31.816001: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9837 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-15 19:20:38.979139: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-15 19:20:39.173677: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 3 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -114.41484 max: 147.61562 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 9.632110595703125e-05 nb_pixel_total : 1000 time to create 1 rle with old method : 0.0019965171813964844 length of segment : 39 time for calcul the mask position with numpy : 6.175041198730469e-05 nb_pixel_total : 897 time to create 1 rle with old method : 0.0017499923706054688 length of segment : 38 time for calcul the mask position with numpy : 5.078315734863281e-05 nb_pixel_total : 55 time to create 1 rle with old method : 0.00013899803161621094 length of segment : 24 time for calcul the mask position with numpy : 4.482269287109375e-05 nb_pixel_total : 36 time to create 1 rle with old method : 0.00013899803161621094 length of segment : 6 time for calcul the mask position with numpy : 4.315376281738281e-05 nb_pixel_total : 67 time to create 1 rle with old method : 0.0001811981201171875 length of segment : 16 time for calcul the mask position with numpy : 4.7206878662109375e-05 nb_pixel_total : 16 time to create 1 rle with old method : 7.128715515136719e-05 length of segment : 6 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 42 time to create 1 rle with old method : 0.00012636184692382812 length of segment : 18 time for calcul the mask position with numpy : 5.0067901611328125e-05 nb_pixel_total : 4 time to create 1 rle with old method : 3.790855407714844e-05 length of segment : 3 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -118.36797 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 4.57763671875e-05 nb_pixel_total : 294 time to create 1 rle with old method : 0.0004374980926513672 length of segment : 28 time for calcul the mask position with numpy : 6.508827209472656e-05 nb_pixel_total : 909 time to create 1 rle with old method : 0.001379251480102539 length of segment : 38 time for calcul the mask position with numpy : 7.2479248046875e-05 nb_pixel_total : 2527 time to create 1 rle with old method : 0.00344085693359375 length of segment : 52 time for calcul the mask position with numpy : 8.440017700195312e-05 nb_pixel_total : 2240 time to create 1 rle with old method : 0.003063678741455078 length of segment : 100 Processing 1 images image shape: (400, 400, 3) min: 42.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -81.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 0.0001277923583984375 nb_pixel_total : 3607 time to create 1 rle with old method : 0.0047092437744140625 length of segment : 93 time for calcul the mask position with numpy : 0.0002677440643310547 nb_pixel_total : 8918 time to create 1 rle with old method : 0.01199030876159668 length of segment : 88 time for calcul the mask position with numpy : 6.67572021484375e-05 nb_pixel_total : 1764 time to create 1 rle with old method : 0.0024781227111816406 length of segment : 47 time for calcul the mask position with numpy : 6.818771362304688e-05 nb_pixel_total : 1974 time to create 1 rle with old method : 0.003094911575317383 length of segment : 25 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -122.88750 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 6.270408630371094e-05 nb_pixel_total : 507 time to create 1 rle with old method : 0.0011208057403564453 length of segment : 29 time for calcul the mask position with numpy : 5.698204040527344e-05 nb_pixel_total : 640 time to create 1 rle with old method : 0.001135110855102539 length of segment : 45 time for calcul the mask position with numpy : 4.315376281738281e-05 nb_pixel_total : 194 time to create 1 rle with old method : 0.0003936290740966797 length of segment : 26 time for calcul the mask position with numpy : 5.364418029785156e-05 nb_pixel_total : 323 time to create 1 rle with old method : 0.0006325244903564453 length of segment : 70 time for calcul the mask position with numpy : 8.249282836914062e-05 nb_pixel_total : 1526 time to create 1 rle with old method : 0.0027511119842529297 length of segment : 75 time for calcul the mask position with numpy : 6.365776062011719e-05 nb_pixel_total : 707 time to create 1 rle with old method : 0.0013706684112548828 length of segment : 44 time for calcul the mask position with numpy : 7.486343383789062e-05 nb_pixel_total : 1570 time to create 1 rle with old method : 0.0028007030487060547 length of segment : 77 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 225.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 96.09219 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 4.1484832763671875e-05 nb_pixel_total : 476 time to create 1 rle with old method : 0.0007321834564208984 length of segment : 18 time for calcul the mask position with numpy : 5.221366882324219e-05 nb_pixel_total : 595 time to create 1 rle with old method : 0.0007231235504150391 length of segment : 72 time for calcul the mask position with numpy : 0.0006680488586425781 nb_pixel_total : 62867 time to create 1 rle with old method : 0.07613134384155273 length of segment : 263 time for calcul the mask position with numpy : 6.198883056640625e-05 nb_pixel_total : 1293 time to create 1 rle with old method : 0.0016798973083496094 length of segment : 47 time for calcul the mask position with numpy : 0.000110626220703125 nb_pixel_total : 508 time to create 1 rle with old method : 0.0009126663208007812 length of segment : 14 Processing 1 images image shape: (400, 400, 3) min: 23.00000 max: 201.00000 molded_images shape: (1, 640, 640, 3) min: -88.25469 max: 83.93203 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -112.66484 max: 144.11953 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 6.866455078125e-05 nb_pixel_total : 2338 time to create 1 rle with old method : 0.0030105113983154297 length of segment : 51 time for calcul the mask position with numpy : 5.078315734863281e-05 nb_pixel_total : 1174 time to create 1 rle with old method : 0.001542806625366211 length of segment : 29 time for calcul the mask position with numpy : 4.696846008300781e-05 nb_pixel_total : 677 time to create 1 rle with old method : 0.0009591579437255859 length of segment : 27 time for calcul the mask position with numpy : 3.0994415283203125e-05 nb_pixel_total : 179 time to create 1 rle with old method : 0.0002713203430175781 length of segment : 19 time for calcul the mask position with numpy : 3.4332275390625e-05 nb_pixel_total : 386 time to create 1 rle with old method : 0.0006384849548339844 length of segment : 24 time for calcul the mask position with numpy : 7.152557373046875e-05 nb_pixel_total : 433 time to create 1 rle with old method : 0.0005698204040527344 length of segment : 46 time for calcul the mask position with numpy : 3.361701965332031e-05 nb_pixel_total : 374 time to create 1 rle with old method : 0.0005216598510742188 length of segment : 24 time for calcul the mask position with numpy : 4.458427429199219e-05 nb_pixel_total : 1207 time to create 1 rle with old method : 0.0016124248504638672 length of segment : 31 time for calcul the mask position with numpy : 3.4809112548828125e-05 nb_pixel_total : 389 time to create 1 rle with old method : 0.0005536079406738281 length of segment : 25 Processing 1 images image shape: (400, 400, 3) min: 18.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -96.78594 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 10 time for calcul the mask position with numpy : 4.935264587402344e-05 nb_pixel_total : 499 time to create 1 rle with old method : 0.0006334781646728516 length of segment : 43 time for calcul the mask position with numpy : 8.940696716308594e-05 nb_pixel_total : 3778 time to create 1 rle with old method : 0.004809141159057617 length of segment : 70 time for calcul the mask position with numpy : 7.152557373046875e-05 nb_pixel_total : 3004 time to create 1 rle with old method : 0.0036439895629882812 length of segment : 57 time for calcul the mask position with numpy : 5.245208740234375e-05 nb_pixel_total : 1579 time to create 1 rle with old method : 0.0019445419311523438 length of segment : 48 time for calcul the mask position with numpy : 3.7670135498046875e-05 nb_pixel_total : 516 time to create 1 rle with old method : 0.0007903575897216797 length of segment : 19 time for calcul the mask position with numpy : 3.266334533691406e-05 nb_pixel_total : 226 time to create 1 rle with old method : 0.0003342628479003906 length of segment : 16 time for calcul the mask position with numpy : 7.510185241699219e-05 nb_pixel_total : 2250 time to create 1 rle with old method : 0.0028429031372070312 length of segment : 73 time for calcul the mask position with numpy : 7.176399230957031e-05 nb_pixel_total : 3133 time to create 1 rle with old method : 0.0040149688720703125 length of segment : 78 time for calcul the mask position with numpy : 4.863739013671875e-05 nb_pixel_total : 589 time to create 1 rle with old method : 0.0008101463317871094 length of segment : 47 time for calcul the mask position with numpy : 3.743171691894531e-05 nb_pixel_total : 513 time to create 1 rle with old method : 0.0007290840148925781 length of segment : 19 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -119.50469 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 8.821487426757812e-05 nb_pixel_total : 1720 time to create 1 rle with old method : 0.0021767616271972656 length of segment : 130 time for calcul the mask position with numpy : 5.626678466796875e-05 nb_pixel_total : 1303 time to create 1 rle with old method : 0.0018367767333984375 length of segment : 39 time for calcul the mask position with numpy : 3.170967102050781e-05 nb_pixel_total : 107 time to create 1 rle with old method : 0.00018405914306640625 length of segment : 10 time for calcul the mask position with numpy : 5.221366882324219e-05 nb_pixel_total : 755 time to create 1 rle with old method : 0.0009434223175048828 length of segment : 37 time for calcul the mask position with numpy : 0.000537872314453125 nb_pixel_total : 31359 time to create 1 rle with old method : 0.03578543663024902 length of segment : 315 time for calcul the mask position with numpy : 0.00011372566223144531 nb_pixel_total : 56 time to create 1 rle with old method : 0.0001800060272216797 length of segment : 26 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 200.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 81.11562 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 6.365776062011719e-05 nb_pixel_total : 1929 time to create 1 rle with old method : 0.0025031566619873047 length of segment : 62 time for calcul the mask position with numpy : 8.630752563476562e-05 nb_pixel_total : 1647 time to create 1 rle with old method : 0.0021729469299316406 length of segment : 158 time for calcul the mask position with numpy : 3.4809112548828125e-05 nb_pixel_total : 195 time to create 1 rle with old method : 0.0002770423889160156 length of segment : 26 time for calcul the mask position with numpy : 0.0001544952392578125 nb_pixel_total : 3498 time to create 1 rle with old method : 0.004425764083862305 length of segment : 181 time for calcul the mask position with numpy : 9.489059448242188e-05 nb_pixel_total : 1335 time to create 1 rle with old method : 0.001874685287475586 length of segment : 146 time for calcul the mask position with numpy : 4.673004150390625e-05 nb_pixel_total : 916 time to create 1 rle with old method : 0.0012199878692626953 length of segment : 33 time for calcul the mask position with numpy : 0.0001742839813232422 nb_pixel_total : 3794 time to create 1 rle with old method : 0.004627704620361328 length of segment : 218 Processing 1 images image shape: (280, 400, 3) min: 18.00000 max: 209.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 80.99844 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.94766 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 11 time for calcul the mask position with numpy : 6.794929504394531e-05 nb_pixel_total : 2772 time to create 1 rle with old method : 0.0037140846252441406 length of segment : 61 time for calcul the mask position with numpy : 4.792213439941406e-05 nb_pixel_total : 1042 time to create 1 rle with old method : 0.0013890266418457031 length of segment : 37 time for calcul the mask position with numpy : 3.790855407714844e-05 nb_pixel_total : 412 time to create 1 rle with old method : 0.0006000995635986328 length of segment : 27 time for calcul the mask position with numpy : 3.0994415283203125e-05 nb_pixel_total : 149 time to create 1 rle with old method : 0.00021195411682128906 length of segment : 23 time for calcul the mask position with numpy : 4.553794860839844e-05 nb_pixel_total : 1057 time to create 1 rle with old method : 0.001317739486694336 length of segment : 52 time for calcul the mask position with numpy : 3.457069396972656e-05 nb_pixel_total : 357 time to create 1 rle with old method : 0.0005099773406982422 length of segment : 27 time for calcul the mask position with numpy : 4.792213439941406e-05 nb_pixel_total : 756 time to create 1 rle with old method : 0.0010149478912353516 length of segment : 32 time for calcul the mask position with numpy : 3.0279159545898438e-05 nb_pixel_total : 99 time to create 1 rle with old method : 0.0001590251922607422 length of segment : 14 time for calcul the mask position with numpy : 3.9577484130859375e-05 nb_pixel_total : 735 time to create 1 rle with old method : 0.0009446144104003906 length of segment : 32 time for calcul the mask position with numpy : 4.863739013671875e-05 nb_pixel_total : 713 time to create 1 rle with old method : 0.0009005069732666016 length of segment : 48 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 699 time to create 1 rle with old method : 0.0009686946868896484 length of segment : 29 Processing 1 images image shape: (280, 400, 3) min: 2.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 138.44375 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 21 time for calcul the mask position with numpy : 0.00010323524475097656 nb_pixel_total : 304 time to create 1 rle with old method : 0.0007119178771972656 length of segment : 21 time for calcul the mask position with numpy : 6.937980651855469e-05 nb_pixel_total : 810 time to create 1 rle with old method : 0.0017130374908447266 length of segment : 48 time for calcul the mask position with numpy : 5.221366882324219e-05 nb_pixel_total : 303 time to create 1 rle with old method : 0.0006527900695800781 length of segment : 24 time for calcul the mask position with numpy : 5.507469177246094e-05 nb_pixel_total : 497 time to create 1 rle with old method : 0.0010533332824707031 length of segment : 30 time for calcul the mask position with numpy : 3.62396240234375e-05 nb_pixel_total : 392 time to create 1 rle with old method : 0.0005497932434082031 length of segment : 21 time for calcul the mask position with numpy : 3.4332275390625e-05 nb_pixel_total : 330 time to create 1 rle with old method : 0.00044918060302734375 length of segment : 28 time for calcul the mask position with numpy : 2.86102294921875e-05 nb_pixel_total : 78 time to create 1 rle with old method : 0.00013828277587890625 length of segment : 7 time for calcul the mask position with numpy : 6.556510925292969e-05 nb_pixel_total : 2033 time to create 1 rle with old method : 0.0025320053100585938 length of segment : 43 time for calcul the mask position with numpy : 4.0531158447265625e-05 nb_pixel_total : 496 time to create 1 rle with old method : 0.0006644725799560547 length of segment : 35 time for calcul the mask position with numpy : 4.220008850097656e-05 nb_pixel_total : 351 time to create 1 rle with old method : 0.0006263256072998047 length of segment : 21 time for calcul the mask position with numpy : 5.221366882324219e-05 nb_pixel_total : 606 time to create 1 rle with old method : 0.0008528232574462891 length of segment : 32 time for calcul the mask position with numpy : 3.266334533691406e-05 nb_pixel_total : 238 time to create 1 rle with old method : 0.00035452842712402344 length of segment : 17 time for calcul the mask position with numpy : 3.0517578125e-05 nb_pixel_total : 254 time to create 1 rle with old method : 0.0003795623779296875 length of segment : 15 time for calcul the mask position with numpy : 4.553794860839844e-05 nb_pixel_total : 693 time to create 1 rle with old method : 0.0009894371032714844 length of segment : 28 time for calcul the mask position with numpy : 3.743171691894531e-05 nb_pixel_total : 413 time to create 1 rle with old method : 0.0005712509155273438 length of segment : 27 time for calcul the mask position with numpy : 3.24249267578125e-05 nb_pixel_total : 205 time to create 1 rle with old method : 0.00030493736267089844 length of segment : 17 time for calcul the mask position with numpy : 2.765655517578125e-05 nb_pixel_total : 53 time to create 1 rle with old method : 9.5367431640625e-05 length of segment : 8 time for calcul the mask position with numpy : 3.24249267578125e-05 nb_pixel_total : 365 time to create 1 rle with old method : 0.0005075931549072266 length of segment : 20 time for calcul the mask position with numpy : 3.933906555175781e-05 nb_pixel_total : 637 time to create 1 rle with old method : 0.0009012222290039062 length of segment : 31 time for calcul the mask position with numpy : 7.677078247070312e-05 nb_pixel_total : 2365 time to create 1 rle with old method : 0.0029382705688476562 length of segment : 65 time for calcul the mask position with numpy : 4.601478576660156e-05 nb_pixel_total : 968 time to create 1 rle with old method : 0.0012972354888916016 length of segment : 51 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 144.91250 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 4.649162292480469e-05 nb_pixel_total : 386 time to create 1 rle with old method : 0.0005161762237548828 length of segment : 32 time for calcul the mask position with numpy : 3.4809112548828125e-05 nb_pixel_total : 283 time to create 1 rle with old method : 0.0003933906555175781 length of segment : 27 time for calcul the mask position with numpy : 5.1021575927734375e-05 nb_pixel_total : 354 time to create 1 rle with old method : 0.0005905628204345703 length of segment : 65 time for calcul the mask position with numpy : 3.695487976074219e-05 nb_pixel_total : 273 time to create 1 rle with old method : 0.00045752525329589844 length of segment : 11 time for calcul the mask position with numpy : 4.1484832763671875e-05 nb_pixel_total : 632 time to create 1 rle with old method : 0.000881195068359375 length of segment : 29 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 132.03750 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 5.53131103515625e-05 nb_pixel_total : 18 time to create 1 rle with old method : 9.799003601074219e-05 length of segment : 4 time for calcul the mask position with numpy : 0.0007948875427246094 nb_pixel_total : 31902 time to create 1 rle with old method : 0.039279937744140625 length of segment : 224 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -115.04766 max: 148.15469 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -118.54766 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 6.699562072753906e-05 nb_pixel_total : 2387 time to create 1 rle with old method : 0.003025054931640625 length of segment : 51 Processing 1 images image shape: (400, 400, 3) min: 41.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -77.79375 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 0.00012874603271484375 nb_pixel_total : 3253 time to create 1 rle with old method : 0.004929780960083008 length of segment : 94 time for calcul the mask position with numpy : 0.0001266002655029297 nb_pixel_total : 7566 time to create 1 rle with old method : 0.009220123291015625 length of segment : 84 time for calcul the mask position with numpy : 8.0108642578125e-05 nb_pixel_total : 1800 time to create 1 rle with old method : 0.0023653507232666016 length of segment : 51 time for calcul the mask position with numpy : 4.9114227294921875e-05 nb_pixel_total : 1500 time to create 1 rle with old method : 0.002040386199951172 length of segment : 21 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -122.88750 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 4.744529724121094e-05 nb_pixel_total : 270 time to create 1 rle with old method : 0.0004754066467285156 length of segment : 29 time for calcul the mask position with numpy : 3.647804260253906e-05 nb_pixel_total : 283 time to create 1 rle with old method : 0.0003724098205566406 length of segment : 51 time for calcul the mask position with numpy : 4.410743713378906e-05 nb_pixel_total : 495 time to create 1 rle with old method : 0.0008261203765869141 length of segment : 29 time for calcul the mask position with numpy : 5.888938903808594e-05 nb_pixel_total : 1538 time to create 1 rle with old method : 0.0019617080688476562 length of segment : 75 time for calcul the mask position with numpy : 4.076957702636719e-05 nb_pixel_total : 103 time to create 1 rle with old method : 0.00015592575073242188 length of segment : 28 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 236.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 105.25625 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 6.532669067382812e-05 nb_pixel_total : 605 time to create 1 rle with old method : 0.0008232593536376953 length of segment : 67 time for calcul the mask position with numpy : 3.814697265625e-05 nb_pixel_total : 502 time to create 1 rle with old method : 0.0007462501525878906 length of segment : 19 time for calcul the mask position with numpy : 3.170967102050781e-05 nb_pixel_total : 125 time to create 1 rle with old method : 0.0002181529998779297 length of segment : 11 time for calcul the mask position with numpy : 8.296966552734375e-05 nb_pixel_total : 1807 time to create 1 rle with old method : 0.0022649765014648438 length of segment : 52 time for calcul the mask position with numpy : 0.0007493495941162109 nb_pixel_total : 67368 time to create 1 rle with old method : 0.07507586479187012 length of segment : 296 Processing 1 images image shape: (400, 400, 3) min: 22.00000 max: 202.00000 molded_images shape: (1, 640, 640, 3) min: -86.84453 max: 84.74453 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -112.66484 max: 142.67422 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 10 time for calcul the mask position with numpy : 9.5367431640625e-05 nb_pixel_total : 2411 time to create 1 rle with old method : 0.003126859664916992 length of segment : 51 time for calcul the mask position with numpy : 4.839897155761719e-05 nb_pixel_total : 1092 time to create 1 rle with old method : 0.0014405250549316406 length of segment : 29 time for calcul the mask position with numpy : 4.076957702636719e-05 nb_pixel_total : 580 time to create 1 rle with old method : 0.000804901123046875 length of segment : 24 time for calcul the mask position with numpy : 3.218650817871094e-05 nb_pixel_total : 182 time to create 1 rle with old method : 0.00029206275939941406 length of segment : 20 time for calcul the mask position with numpy : 4.00543212890625e-05 nb_pixel_total : 554 time to create 1 rle with old method : 0.0007414817810058594 length of segment : 24 time for calcul the mask position with numpy : 4.553794860839844e-05 nb_pixel_total : 607 time to create 1 rle with old method : 0.0009806156158447266 length of segment : 35 time for calcul the mask position with numpy : 7.677078247070312e-05 nb_pixel_total : 241 time to create 1 rle with old method : 0.0003294944763183594 length of segment : 39 time for calcul the mask position with numpy : 3.7670135498046875e-05 nb_pixel_total : 418 time to create 1 rle with old method : 0.000579833984375 length of segment : 25 time for calcul the mask position with numpy : 3.1948089599609375e-05 nb_pixel_total : 224 time to create 1 rle with old method : 0.0003151893615722656 length of segment : 23 time for calcul the mask position with numpy : 3.457069396972656e-05 nb_pixel_total : 409 time to create 1 rle with old method : 0.0005638599395751953 length of segment : 26 Processing 1 images image shape: (400, 400, 3) min: 17.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -93.43828 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 4.76837158203125e-05 nb_pixel_total : 177 time to create 1 rle with old method : 0.0004363059997558594 length of segment : 15 time for calcul the mask position with numpy : 9.417533874511719e-05 nb_pixel_total : 4118 time to create 1 rle with old method : 0.00527191162109375 length of segment : 118 time for calcul the mask position with numpy : 3.1948089599609375e-05 nb_pixel_total : 94 time to create 1 rle with old method : 0.0001499652862548828 length of segment : 10 time for calcul the mask position with numpy : 3.218650817871094e-05 nb_pixel_total : 344 time to create 1 rle with old method : 0.0005350112915039062 length of segment : 12 time for calcul the mask position with numpy : 3.123283386230469e-05 nb_pixel_total : 246 time to create 1 rle with old method : 0.0003445148468017578 length of segment : 18 time for calcul the mask position with numpy : 3.314018249511719e-05 nb_pixel_total : 230 time to create 1 rle with old method : 0.00036025047302246094 length of segment : 16 time for calcul the mask position with numpy : 8.416175842285156e-05 nb_pixel_total : 4059 time to create 1 rle with old method : 0.005253791809082031 length of segment : 62 time for calcul the mask position with numpy : 4.1961669921875e-05 nb_pixel_total : 508 time to create 1 rle with old method : 0.0007123947143554688 length of segment : 49 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -118.56719 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 0.00011444091796875 nb_pixel_total : 1837 time to create 1 rle with old method : 0.0022640228271484375 length of segment : 124 time for calcul the mask position with numpy : 5.245208740234375e-05 nb_pixel_total : 1248 time to create 1 rle with old method : 0.0015902519226074219 length of segment : 38 time for calcul the mask position with numpy : 4.482269287109375e-05 nb_pixel_total : 844 time to create 1 rle with old method : 0.001069784164428711 length of segment : 42 time for calcul the mask position with numpy : 0.0005104541778564453 nb_pixel_total : 33535 time to create 1 rle with old method : 0.03710341453552246 length of segment : 345 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 336 time to create 1 rle with old method : 0.00048160552978515625 length of segment : 29 time for calcul the mask position with numpy : 0.0005190372467041016 nb_pixel_total : 40375 time to create 1 rle with old method : 0.04486489295959473 length of segment : 252 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 198.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 79.75625 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 7.200241088867188e-05 nb_pixel_total : 1690 time to create 1 rle with old method : 0.0021250247955322266 length of segment : 40 time for calcul the mask position with numpy : 8.416175842285156e-05 nb_pixel_total : 1808 time to create 1 rle with old method : 0.002318143844604492 length of segment : 175 time for calcul the mask position with numpy : 8.130073547363281e-05 nb_pixel_total : 1683 time to create 1 rle with old method : 0.0022079944610595703 length of segment : 165 time for calcul the mask position with numpy : 4.2438507080078125e-05 nb_pixel_total : 528 time to create 1 rle with old method : 0.0007374286651611328 length of segment : 33 time for calcul the mask position with numpy : 3.314018249511719e-05 nb_pixel_total : 191 time to create 1 rle with old method : 0.0002703666687011719 length of segment : 28 time for calcul the mask position with numpy : 0.0001811981201171875 nb_pixel_total : 4014 time to create 1 rle with old method : 0.004901885986328125 length of segment : 188 time for calcul the mask position with numpy : 0.0001785755157470703 nb_pixel_total : 4065 time to create 1 rle with old method : 0.007039785385131836 length of segment : 212 time for calcul the mask position with numpy : 8.440017700195312e-05 nb_pixel_total : 1176 time to create 1 rle with old method : 0.0018489360809326172 length of segment : 79 time for calcul the mask position with numpy : 0.00023937225341796875 nb_pixel_total : 4581 time to create 1 rle with old method : 0.0061719417572021484 length of segment : 224 Processing 1 images image shape: (280, 400, 3) min: 25.00000 max: 204.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 76.38125 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.000957489013671875 nb_pixel_total : 105553 time to create 1 rle with old method : 0.12069368362426758 length of segment : 283 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.93984 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 13 time for calcul the mask position with numpy : 6.818771362304688e-05 nb_pixel_total : 2255 time to create 1 rle with old method : 0.0029408931732177734 length of segment : 66 time for calcul the mask position with numpy : 0.00010371208190917969 nb_pixel_total : 3748 time to create 1 rle with old method : 0.004514932632446289 length of segment : 92 time for calcul the mask position with numpy : 3.504753112792969e-05 nb_pixel_total : 229 time to create 1 rle with old method : 0.0003204345703125 length of segment : 30 time for calcul the mask position with numpy : 4.9591064453125e-05 nb_pixel_total : 1008 time to create 1 rle with old method : 0.0012888908386230469 length of segment : 37 time for calcul the mask position with numpy : 3.8623809814453125e-05 nb_pixel_total : 502 time to create 1 rle with old method : 0.0007011890411376953 length of segment : 31 time for calcul the mask position with numpy : 2.956390380859375e-05 nb_pixel_total : 79 time to create 1 rle with old method : 0.0001437664031982422 length of segment : 10 time for calcul the mask position with numpy : 3.337860107421875e-05 nb_pixel_total : 352 time to create 1 rle with old method : 0.00048732757568359375 length of segment : 29 time for calcul the mask position with numpy : 4.553794860839844e-05 nb_pixel_total : 1019 time to create 1 rle with old method : 0.001323699951171875 length of segment : 28 time for calcul the mask position with numpy : 4.6253204345703125e-05 nb_pixel_total : 901 time to create 1 rle with old method : 0.0012056827545166016 length of segment : 36 time for calcul the mask position with numpy : 3.337860107421875e-05 nb_pixel_total : 326 time to create 1 rle with old method : 0.0004684925079345703 length of segment : 20 time for calcul the mask position with numpy : 4.363059997558594e-05 nb_pixel_total : 666 time to create 1 rle with old method : 0.0008320808410644531 length of segment : 49 time for calcul the mask position with numpy : 3.647804260253906e-05 nb_pixel_total : 369 time to create 1 rle with old method : 0.0005090236663818359 length of segment : 28 time for calcul the mask position with numpy : 4.3392181396484375e-05 nb_pixel_total : 813 time to create 1 rle with old method : 0.0011496543884277344 length of segment : 35 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 136.45547 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 20 time for calcul the mask position with numpy : 7.081031799316406e-05 nb_pixel_total : 264 time to create 1 rle with old method : 0.0008106231689453125 length of segment : 22 time for calcul the mask position with numpy : 5.364418029785156e-05 nb_pixel_total : 880 time to create 1 rle with old method : 0.0013318061828613281 length of segment : 57 time for calcul the mask position with numpy : 3.981590270996094e-05 nb_pixel_total : 351 time to create 1 rle with old method : 0.0005526542663574219 length of segment : 19 time for calcul the mask position with numpy : 4.57763671875e-05 nb_pixel_total : 508 time to create 1 rle with old method : 0.0007550716400146484 length of segment : 31 time for calcul the mask position with numpy : 4.1484832763671875e-05 nb_pixel_total : 276 time to create 1 rle with old method : 0.00042176246643066406 length of segment : 21 time for calcul the mask position with numpy : 4.482269287109375e-05 nb_pixel_total : 458 time to create 1 rle with old method : 0.0007233619689941406 length of segment : 23 time for calcul the mask position with numpy : 5.507469177246094e-05 nb_pixel_total : 779 time to create 1 rle with old method : 0.0012540817260742188 length of segment : 33 time for calcul the mask position with numpy : 4.6253204345703125e-05 nb_pixel_total : 434 time to create 1 rle with old method : 0.0006673336029052734 length of segment : 26 time for calcul the mask position with numpy : 4.458427429199219e-05 nb_pixel_total : 223 time to create 1 rle with old method : 0.00040268898010253906 length of segment : 21 time for calcul the mask position with numpy : 4.315376281738281e-05 nb_pixel_total : 196 time to create 1 rle with old method : 0.0003330707550048828 length of segment : 19 time for calcul the mask position with numpy : 4.696846008300781e-05 nb_pixel_total : 360 time to create 1 rle with old method : 0.0005810260772705078 length of segment : 43 time for calcul the mask position with numpy : 2.9087066650390625e-05 nb_pixel_total : 79 time to create 1 rle with old method : 0.0001513957977294922 length of segment : 7 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 541 time to create 1 rle with old method : 0.0006923675537109375 length of segment : 38 time for calcul the mask position with numpy : 4.267692565917969e-05 nb_pixel_total : 620 time to create 1 rle with old method : 0.0008392333984375 length of segment : 36 time for calcul the mask position with numpy : 6.771087646484375e-05 nb_pixel_total : 2021 time to create 1 rle with old method : 0.0026743412017822266 length of segment : 39 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 270 time to create 1 rle with old method : 0.0004451274871826172 length of segment : 17 time for calcul the mask position with numpy : 5.9604644775390625e-05 nb_pixel_total : 2053 time to create 1 rle with old method : 0.002759695053100586 length of segment : 40 time for calcul the mask position with numpy : 4.458427429199219e-05 nb_pixel_total : 205 time to create 1 rle with old method : 0.00038170814514160156 length of segment : 17 time for calcul the mask position with numpy : 3.0040740966796875e-05 nb_pixel_total : 188 time to create 1 rle with old method : 0.0003197193145751953 length of segment : 16 time for calcul the mask position with numpy : 3.147125244140625e-05 nb_pixel_total : 250 time to create 1 rle with old method : 0.0003612041473388672 length of segment : 18 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 147.54141 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 8.130073547363281e-05 nb_pixel_total : 1132 time to create 1 rle with old method : 0.0018453598022460938 length of segment : 89 time for calcul the mask position with numpy : 4.839897155761719e-05 nb_pixel_total : 369 time to create 1 rle with old method : 0.0005559921264648438 length of segment : 27 length of segment : 0 time for calcul the mask position with numpy : 6.556510925292969e-05 nb_pixel_total : 421 time to create 1 rle with old method : 0.000759124755859375 length of segment : 28 time for calcul the mask position with numpy : 5.0067901611328125e-05 nb_pixel_total : 642 time to create 1 rle with old method : 0.0010297298431396484 length of segment : 22 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 139.41250 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0004868507385253906 nb_pixel_total : 37026 time to create 1 rle with old method : 0.04745912551879883 length of segment : 222 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -119.90313 max: 147.86562 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 7.05718994140625e-05 nb_pixel_total : 973 time to create 1 rle with old method : 0.0019538402557373047 length of segment : 41 time for calcul the mask position with numpy : 7.271766662597656e-05 nb_pixel_total : 826 time to create 1 rle with old method : 0.001992464065551758 length of segment : 35 time for calcul the mask position with numpy : 5.269050598144531e-05 nb_pixel_total : 71 time to create 1 rle with old method : 0.0001900196075439453 length of segment : 24 time for calcul the mask position with numpy : 5.1975250244140625e-05 nb_pixel_total : 55 time to create 1 rle with old method : 0.00021195411682128906 length of segment : 11 time for calcul the mask position with numpy : 3.0040740966796875e-05 nb_pixel_total : 64 time to create 1 rle with old method : 0.00011968612670898438 length of segment : 28 time for calcul the mask position with numpy : 2.7179718017578125e-05 nb_pixel_total : 4 time to create 1 rle with old method : 2.4080276489257812e-05 length of segment : 2 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.81719 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 6.628036499023438e-05 nb_pixel_total : 2406 time to create 1 rle with old method : 0.0031664371490478516 length of segment : 50 time for calcul the mask position with numpy : 3.337860107421875e-05 nb_pixel_total : 70 time to create 1 rle with old method : 0.00013375282287597656 length of segment : 30 Processing 1 images image shape: (400, 400, 3) min: 42.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -77.94609 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 0.0002086162567138672 nb_pixel_total : 9880 time to create 1 rle with old method : 0.013587474822998047 length of segment : 89 time for calcul the mask position with numpy : 0.00015926361083984375 nb_pixel_total : 4708 time to create 1 rle with old method : 0.006545543670654297 length of segment : 57 time for calcul the mask position with numpy : 9.5367431640625e-05 nb_pixel_total : 1790 time to create 1 rle with old method : 0.0028944015502929688 length of segment : 25 time for calcul the mask position with numpy : 5.936622619628906e-05 nb_pixel_total : 1532 time to create 1 rle with old method : 0.0025205612182617188 length of segment : 22 time for calcul the mask position with numpy : 5.626678466796875e-05 nb_pixel_total : 1285 time to create 1 rle with old method : 0.0020911693572998047 length of segment : 24 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -122.83672 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 4.792213439941406e-05 nb_pixel_total : 519 time to create 1 rle with old method : 0.0009279251098632812 length of segment : 29 time for calcul the mask position with numpy : 4.029273986816406e-05 nb_pixel_total : 563 time to create 1 rle with old method : 0.0007448196411132812 length of segment : 67 time for calcul the mask position with numpy : 4.00543212890625e-05 nb_pixel_total : 229 time to create 1 rle with old method : 0.00039124488830566406 length of segment : 25 time for calcul the mask position with numpy : 7.510185241699219e-05 nb_pixel_total : 1486 time to create 1 rle with old method : 0.002123594284057617 length of segment : 74 time for calcul the mask position with numpy : 7.915496826171875e-05 nb_pixel_total : 1419 time to create 1 rle with old method : 0.0019617080688476562 length of segment : 75 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 240.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 108.17031 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 4.649162292480469e-05 nb_pixel_total : 490 time to create 1 rle with old method : 0.0007812976837158203 length of segment : 17 time for calcul the mask position with numpy : 3.218650817871094e-05 nb_pixel_total : 162 time to create 1 rle with old method : 0.0002570152282714844 length of segment : 13 time for calcul the mask position with numpy : 7.319450378417969e-05 nb_pixel_total : 605 time to create 1 rle with old method : 0.0013217926025390625 length of segment : 72 time for calcul the mask position with numpy : 5.817413330078125e-05 nb_pixel_total : 1835 time to create 1 rle with old method : 0.0022847652435302734 length of segment : 71 time for calcul the mask position with numpy : 0.00012946128845214844 nb_pixel_total : 3522 time to create 1 rle with old method : 0.004700183868408203 length of segment : 153 time for calcul the mask position with numpy : 0.0008084774017333984 nb_pixel_total : 62976 time to create 1 rle with old method : 0.07843589782714844 length of segment : 283 Processing 1 images image shape: (400, 400, 3) min: 24.00000 max: 204.00000 molded_images shape: (1, 640, 640, 3) min: -89.11797 max: 86.80703 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -112.66484 max: 143.18203 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 8.58306884765625e-05 nb_pixel_total : 186 time to create 1 rle with old method : 0.0003561973571777344 length of segment : 20 time for calcul the mask position with numpy : 4.267692565917969e-05 nb_pixel_total : 754 time to create 1 rle with old method : 0.001348257064819336 length of segment : 26 time for calcul the mask position with numpy : 5.650520324707031e-05 nb_pixel_total : 1191 time to create 1 rle with old method : 0.0024209022521972656 length of segment : 25 time for calcul the mask position with numpy : 5.53131103515625e-05 nb_pixel_total : 396 time to create 1 rle with old method : 0.0008416175842285156 length of segment : 24 time for calcul the mask position with numpy : 0.00010037422180175781 nb_pixel_total : 2438 time to create 1 rle with old method : 0.0034601688385009766 length of segment : 52 time for calcul the mask position with numpy : 5.14984130859375e-05 nb_pixel_total : 1289 time to create 1 rle with old method : 0.001821279525756836 length of segment : 30 time for calcul the mask position with numpy : 0.0001163482666015625 nb_pixel_total : 1667 time to create 1 rle with old method : 0.0023779869079589844 length of segment : 111 time for calcul the mask position with numpy : 4.38690185546875e-05 nb_pixel_total : 392 time to create 1 rle with old method : 0.0006568431854248047 length of segment : 21 time for calcul the mask position with numpy : 3.600120544433594e-05 nb_pixel_total : 399 time to create 1 rle with old method : 0.0006048679351806641 length of segment : 25 Processing 1 images image shape: (400, 400, 3) min: 16.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -94.65313 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 10 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 529 time to create 1 rle with old method : 0.0007855892181396484 length of segment : 20 time for calcul the mask position with numpy : 9.322166442871094e-05 nb_pixel_total : 380 time to create 1 rle with old method : 0.0008473396301269531 length of segment : 28 time for calcul the mask position with numpy : 9.417533874511719e-05 nb_pixel_total : 3896 time to create 1 rle with old method : 0.005594491958618164 length of segment : 70 time for calcul the mask position with numpy : 5.6743621826171875e-05 nb_pixel_total : 229 time to create 1 rle with old method : 0.00042128562927246094 length of segment : 16 time for calcul the mask position with numpy : 0.00012111663818359375 nb_pixel_total : 3905 time to create 1 rle with old method : 0.006110191345214844 length of segment : 59 time for calcul the mask position with numpy : 5.4836273193359375e-05 nb_pixel_total : 156 time to create 1 rle with old method : 0.0003063678741455078 length of segment : 16 time for calcul the mask position with numpy : 0.00012087821960449219 nb_pixel_total : 1967 time to create 1 rle with old method : 0.0029799938201904297 length of segment : 59 time for calcul the mask position with numpy : 9.250640869140625e-05 nb_pixel_total : 2870 time to create 1 rle with old method : 0.0042536258697509766 length of segment : 58 time for calcul the mask position with numpy : 5.650520324707031e-05 nb_pixel_total : 347 time to create 1 rle with old method : 0.0006511211395263672 length of segment : 12 time for calcul the mask position with numpy : 0.0001251697540283203 nb_pixel_total : 4495 time to create 1 rle with old method : 0.0059032440185546875 length of segment : 67 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -118.82109 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 0.00010395050048828125 nb_pixel_total : 1484 time to create 1 rle with old method : 0.0019025802612304688 length of segment : 113 time for calcul the mask position with numpy : 5.626678466796875e-05 nb_pixel_total : 1275 time to create 1 rle with old method : 0.001680612564086914 length of segment : 41 time for calcul the mask position with numpy : 0.0005507469177246094 nb_pixel_total : 38954 time to create 1 rle with old method : 0.0449979305267334 length of segment : 219 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 218 time to create 1 rle with old method : 0.0003409385681152344 length of segment : 24 time for calcul the mask position with numpy : 3.075599670410156e-05 nb_pixel_total : 108 time to create 1 rle with old method : 0.0001914501190185547 length of segment : 11 time for calcul the mask position with numpy : 0.00017905235290527344 nb_pixel_total : 1783 time to create 1 rle with old method : 0.0026543140411376953 length of segment : 127 time for calcul the mask position with numpy : 4.363059997558594e-05 nb_pixel_total : 795 time to create 1 rle with old method : 0.0010297298431396484 length of segment : 38 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 200.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 81.11562 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 10 time for calcul the mask position with numpy : 7.43865966796875e-05 nb_pixel_total : 2002 time to create 1 rle with old method : 0.0026006698608398438 length of segment : 54 time for calcul the mask position with numpy : 8.654594421386719e-05 nb_pixel_total : 2476 time to create 1 rle with old method : 0.0031290054321289062 length of segment : 203 time for calcul the mask position with numpy : 3.409385681152344e-05 nb_pixel_total : 197 time to create 1 rle with old method : 0.0002925395965576172 length of segment : 25 time for calcul the mask position with numpy : 9.560585021972656e-05 nb_pixel_total : 3334 time to create 1 rle with old method : 0.004149675369262695 length of segment : 175 time for calcul the mask position with numpy : 5.555152893066406e-05 nb_pixel_total : 986 time to create 1 rle with old method : 0.0012562274932861328 length of segment : 53 time for calcul the mask position with numpy : 3.838539123535156e-05 nb_pixel_total : 540 time to create 1 rle with old method : 0.0007278919219970703 length of segment : 37 time for calcul the mask position with numpy : 0.0001761913299560547 nb_pixel_total : 4007 time to create 1 rle with old method : 0.005079507827758789 length of segment : 191 time for calcul the mask position with numpy : 6.818771362304688e-05 nb_pixel_total : 1330 time to create 1 rle with old method : 0.0016939640045166016 length of segment : 80 time for calcul the mask position with numpy : 0.00018525123596191406 nb_pixel_total : 4461 time to create 1 rle with old method : 0.0052165985107421875 length of segment : 227 time for calcul the mask position with numpy : 5.698204040527344e-05 nb_pixel_total : 1254 time to create 1 rle with old method : 0.0016198158264160156 length of segment : 77 Processing 1 images image shape: (280, 400, 3) min: 23.00000 max: 211.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 80.05703 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0009615421295166016 nb_pixel_total : 105529 time to create 1 rle with old method : 0.12401127815246582 length of segment : 284 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.92422 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 5.8650970458984375e-05 nb_pixel_total : 1889 time to create 1 rle with old method : 0.002526521682739258 length of segment : 54 time for calcul the mask position with numpy : 4.696846008300781e-05 nb_pixel_total : 1045 time to create 1 rle with old method : 0.0013766288757324219 length of segment : 37 time for calcul the mask position with numpy : 3.528594970703125e-05 nb_pixel_total : 179 time to create 1 rle with old method : 0.0003113746643066406 length of segment : 26 time for calcul the mask position with numpy : 3.409385681152344e-05 nb_pixel_total : 326 time to create 1 rle with old method : 0.0005164146423339844 length of segment : 27 time for calcul the mask position with numpy : 3.600120544433594e-05 nb_pixel_total : 451 time to create 1 rle with old method : 0.0007233619689941406 length of segment : 30 time for calcul the mask position with numpy : 7.152557373046875e-05 nb_pixel_total : 2937 time to create 1 rle with old method : 0.005512237548828125 length of segment : 61 Processing 1 images image shape: (280, 400, 3) min: 1.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 138.44375 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 19 time for calcul the mask position with numpy : 4.935264587402344e-05 nb_pixel_total : 396 time to create 1 rle with old method : 0.0005769729614257812 length of segment : 29 time for calcul the mask position with numpy : 4.0531158447265625e-05 nb_pixel_total : 495 time to create 1 rle with old method : 0.0006999969482421875 length of segment : 40 time for calcul the mask position with numpy : 3.528594970703125e-05 nb_pixel_total : 395 time to create 1 rle with old method : 0.00054931640625 length of segment : 21 time for calcul the mask position with numpy : 2.86102294921875e-05 nb_pixel_total : 84 time to create 1 rle with old method : 0.00015807151794433594 length of segment : 8 time for calcul the mask position with numpy : 3.814697265625e-05 nb_pixel_total : 478 time to create 1 rle with old method : 0.0006856918334960938 length of segment : 31 time for calcul the mask position with numpy : 4.982948303222656e-05 nb_pixel_total : 531 time to create 1 rle with old method : 0.0007748603820800781 length of segment : 34 time for calcul the mask position with numpy : 6.985664367675781e-05 nb_pixel_total : 1956 time to create 1 rle with old method : 0.0029518604278564453 length of segment : 40 time for calcul the mask position with numpy : 3.743171691894531e-05 nb_pixel_total : 257 time to create 1 rle with old method : 0.0003800392150878906 length of segment : 16 time for calcul the mask position with numpy : 6.651878356933594e-05 nb_pixel_total : 455 time to create 1 rle with old method : 0.0006527900695800781 length of segment : 25 time for calcul the mask position with numpy : 4.363059997558594e-05 nb_pixel_total : 805 time to create 1 rle with old method : 0.0012695789337158203 length of segment : 30 time for calcul the mask position with numpy : 3.266334533691406e-05 nb_pixel_total : 214 time to create 1 rle with old method : 0.0003256797790527344 length of segment : 16 time for calcul the mask position with numpy : 2.8848648071289062e-05 nb_pixel_total : 99 time to create 1 rle with old method : 0.00021791458129882812 length of segment : 10 time for calcul the mask position with numpy : 5.6743621826171875e-05 nb_pixel_total : 1881 time to create 1 rle with old method : 0.002307415008544922 length of segment : 42 time for calcul the mask position with numpy : 3.266334533691406e-05 nb_pixel_total : 265 time to create 1 rle with old method : 0.00038123130798339844 length of segment : 20 time for calcul the mask position with numpy : 3.1948089599609375e-05 nb_pixel_total : 191 time to create 1 rle with old method : 0.00028061866760253906 length of segment : 17 time for calcul the mask position with numpy : 3.457069396972656e-05 nb_pixel_total : 248 time to create 1 rle with old method : 0.0003445148468017578 length of segment : 24 time for calcul the mask position with numpy : 3.7670135498046875e-05 nb_pixel_total : 462 time to create 1 rle with old method : 0.0006442070007324219 length of segment : 35 time for calcul the mask position with numpy : 4.601478576660156e-05 nb_pixel_total : 380 time to create 1 rle with old method : 0.0005726814270019531 length of segment : 24 time for calcul the mask position with numpy : 4.410743713378906e-05 nb_pixel_total : 866 time to create 1 rle with old method : 0.0011222362518310547 length of segment : 46 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 146.51406 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 7.987022399902344e-05 nb_pixel_total : 210 time to create 1 rle with old method : 0.0006096363067626953 length of segment : 43 time for calcul the mask position with numpy : 5.2928924560546875e-05 nb_pixel_total : 275 time to create 1 rle with old method : 0.00042891502380371094 length of segment : 27 time for calcul the mask position with numpy : 4.935264587402344e-05 nb_pixel_total : 355 time to create 1 rle with old method : 0.0006103515625 length of segment : 31 time for calcul the mask position with numpy : 4.601478576660156e-05 nb_pixel_total : 674 time to create 1 rle with old method : 0.0009496212005615234 length of segment : 25 time for calcul the mask position with numpy : 5.030632019042969e-05 nb_pixel_total : 371 time to create 1 rle with old method : 0.0005164146423339844 length of segment : 20 length of segment : 0 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 132.60000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 0.0004324913024902344 nb_pixel_total : 37363 time to create 1 rle with old method : 0.04612278938293457 length of segment : 228 time for calcul the mask position with numpy : 4.076957702636719e-05 nb_pixel_total : 20 time to create 1 rle with old method : 6.365776062011719e-05 length of segment : 4 time for calcul the mask position with numpy : 0.0003864765167236328 nb_pixel_total : 27915 time to create 1 rle with old method : 0.033455610275268555 length of segment : 245 Detection mask done ! Trying to reset tf kernel 1862232 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 4682 tf kernel not reseted sub process len(results) : 282 len(list_Values) 282 None max_time_sub_proc : 3600 parent process len(results) : 0 len(list_Values) 282 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 9971 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Catched exception ! Connect or reconnect ! Number saved : None batch 1 Loaded 282 chid ids of type : 4228 Number RLEs to save : 15525 save missing photos in datou_result : time spend for datou_step_exec : 25.83660101890564 time spend to save output : 1.7434210777282715 total time spend for step 1 : 27.58002209663391 step2:brightness Sat Feb 15 19:20:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1739643628_1861816_1337909766_701c5dedf83cfed937a8b6a881561475.jpg treat image : temp/1739643628_1861816_1337909751_d1d9754e167faa7420659fa87344ec8f.jpg treat image : temp/1739643628_1861816_1337909750_f03f9ec0ecba5f4848ddc278cd08402a.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 3 time used for this insertion : 0.01321101188659668 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 3 time used for this insertion : 0.011133909225463867 save missing photos in datou_result : time spend for datou_step_exec : 0.7552833557128906 time spend to save output : 0.03124213218688965 total time spend for step 2 : 0.7865254878997803 step3:blur_detection Sat Feb 15 19:20:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1739643628_1861816_1337909766_701c5dedf83cfed937a8b6a881561475.jpg resize: (1080, 1920) 1337909766 -7.121482389800247 treat image : temp/1739643628_1861816_1337909751_d1d9754e167faa7420659fa87344ec8f.jpg resize: (1080, 1920) 1337909751 -7.098568046193153 treat image : temp/1739643628_1861816_1337909750_f03f9ec0ecba5f4848ddc278cd08402a.jpg resize: (1080, 1920) 1337909750 -7.108969347317072 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 3 time used for this insertion : 0.012305974960327148 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 3 time used for this insertion : 0.012453556060791016 save missing photos in datou_result : time spend for datou_step_exec : 2.920614242553711 time spend to save output : 0.029492855072021484 total time spend for step 3 : 2.9501070976257324 step4:rle_unique_nms_with_priority Sat Feb 15 19:20:59 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 282 chid ids of type : 4228 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 98 nb_hashtags : 6 time to prepare the origin masks : 1.1119520664215088 time for calcul the mask position with numpy : 0.40012550354003906 nb_pixel_total : 1863932 time to create 1 rle with new method : 0.11970901489257812 time for calcul the mask position with numpy : 0.009105443954467773 nb_pixel_total : 62867 time to create 1 rle with old method : 0.06779623031616211 time for calcul the mask position with numpy : 0.008063793182373047 nb_pixel_total : 55 time to create 1 rle with old method : 6.461143493652344e-05 time for calcul the mask position with numpy : 0.008055686950683594 nb_pixel_total : 1000 time to create 1 rle with old method : 0.0012080669403076172 time for calcul the mask position with numpy : 0.008003711700439453 nb_pixel_total : 42 time to create 1 rle with old method : 5.316734313964844e-05 time for calcul the mask position with numpy : 0.007824420928955078 nb_pixel_total : 294 time to create 1 rle with old method : 0.0003006458282470703 time for calcul the mask position with numpy : 0.008154869079589844 nb_pixel_total : 36 time to create 1 rle with old method : 5.1975250244140625e-05 time for calcul the mask position with numpy : 0.008263111114501953 nb_pixel_total : 3607 time to create 1 rle with old method : 0.004196882247924805 time for calcul the mask position with numpy : 0.008376598358154297 nb_pixel_total : 2527 time to create 1 rle with old method : 0.0029468536376953125 time for calcul the mask position with numpy : 0.008379697799682617 nb_pixel_total : 4 time to create 1 rle with old method : 1.2159347534179688e-05 time for calcul the mask position with numpy : 0.008375883102416992 nb_pixel_total : 23 time to create 1 rle with old method : 5.888938903808594e-05 time for calcul the mask position with numpy : 0.008367538452148438 nb_pixel_total : 640 time to create 1 rle with old method : 0.0007419586181640625 time for calcul the mask position with numpy : 0.008367776870727539 nb_pixel_total : 445 time to create 1 rle with old method : 0.0005319118499755859 time for calcul the mask position with numpy : 0.008317708969116211 nb_pixel_total : 1431 time to create 1 rle with old method : 0.0015988349914550781 time for calcul the mask position with numpy : 0.008358955383300781 nb_pixel_total : 313 time to create 1 rle with old method : 0.00038242340087890625 time for calcul the mask position with numpy : 0.00835728645324707 nb_pixel_total : 67 time to create 1 rle with old method : 8.7738037109375e-05 time for calcul the mask position with numpy : 0.008341312408447266 nb_pixel_total : 595 time to create 1 rle with old method : 0.0006875991821289062 time for calcul the mask position with numpy : 0.008348464965820312 nb_pixel_total : 194 time to create 1 rle with old method : 0.00022292137145996094 time for calcul the mask position with numpy : 0.008372306823730469 nb_pixel_total : 909 time to create 1 rle with old method : 0.0010597705841064453 time for calcul the mask position with numpy : 0.008403778076171875 nb_pixel_total : 8918 time to create 1 rle with old method : 0.010372400283813477 time for calcul the mask position with numpy : 0.00836038589477539 nb_pixel_total : 507 time to create 1 rle with old method : 0.0006244182586669922 time for calcul the mask position with numpy : 0.008348703384399414 nb_pixel_total : 104 time to create 1 rle with old method : 0.00014066696166992188 time for calcul the mask position with numpy : 0.008342981338500977 nb_pixel_total : 1526 time to create 1 rle with old method : 0.0017628669738769531 time for calcul the mask position with numpy : 0.00836944580078125 nb_pixel_total : 1764 time to create 1 rle with old method : 0.002097606658935547 time for calcul the mask position with numpy : 0.008380889892578125 nb_pixel_total : 508 time to create 1 rle with old method : 0.0006511211395263672 time for calcul the mask position with numpy : 0.008356094360351562 nb_pixel_total : 1293 time to create 1 rle with old method : 0.0014955997467041016 time for calcul the mask position with numpy : 0.008435249328613281 nb_pixel_total : 1974 time to create 1 rle with old method : 0.0023217201232910156 time for calcul the mask position with numpy : 0.008376121520996094 nb_pixel_total : 476 time to create 1 rle with old method : 0.0005507469177246094 time for calcul the mask position with numpy : 0.008358955383300781 nb_pixel_total : 589 time to create 1 rle with old method : 0.0006678104400634766 time for calcul the mask position with numpy : 0.008347034454345703 nb_pixel_total : 3133 time to create 1 rle with old method : 0.0036330223083496094 time for calcul the mask position with numpy : 0.008343696594238281 nb_pixel_total : 433 time to create 1 rle with old method : 0.0005080699920654297 time for calcul the mask position with numpy : 0.00836038589477539 nb_pixel_total : 3778 time to create 1 rle with old method : 0.00419163703918457 time for calcul the mask position with numpy : 0.008357048034667969 nb_pixel_total : 49 time to create 1 rle with old method : 7.176399230957031e-05 time for calcul the mask position with numpy : 0.008379697799682617 nb_pixel_total : 755 time to create 1 rle with old method : 0.0009152889251708984 time for calcul the mask position with numpy : 0.008410215377807617 nb_pixel_total : 195 time to create 1 rle with old method : 0.00023603439331054688 time for calcul the mask position with numpy : 0.008376359939575195 nb_pixel_total : 56 time to create 1 rle with old method : 8.821487426757812e-05 time for calcul the mask position with numpy : 0.008408546447753906 nb_pixel_total : 1303 time to create 1 rle with old method : 0.0015254020690917969 time for calcul the mask position with numpy : 0.008647441864013672 nb_pixel_total : 1929 time to create 1 rle with old method : 0.0021979808807373047 time for calcul the mask position with numpy : 0.008876562118530273 nb_pixel_total : 31359 time to create 1 rle with old method : 0.036101341247558594 time for calcul the mask position with numpy : 0.008412361145019531 nb_pixel_total : 916 time to create 1 rle with old method : 0.0010476112365722656 time for calcul the mask position with numpy : 0.008306741714477539 nb_pixel_total : 668 time to create 1 rle with old method : 0.0008161067962646484 time for calcul the mask position with numpy : 0.008347749710083008 nb_pixel_total : 249 time to create 1 rle with old method : 0.0005431175231933594 time for calcul the mask position with numpy : 0.008343219757080078 nb_pixel_total : 1647 time to create 1 rle with old method : 0.001953601837158203 time for calcul the mask position with numpy : 0.00830221176147461 nb_pixel_total : 3498 time to create 1 rle with old method : 0.003945589065551758 time for calcul the mask position with numpy : 0.00830078125 nb_pixel_total : 1720 time to create 1 rle with old method : 0.0020051002502441406 time for calcul the mask position with numpy : 0.008360624313354492 nb_pixel_total : 2338 time to create 1 rle with old method : 0.0027053356170654297 time for calcul the mask position with numpy : 0.008386611938476562 nb_pixel_total : 220 time to create 1 rle with old method : 0.0002703666687011719 time for calcul the mask position with numpy : 0.00843954086303711 nb_pixel_total : 25 time to create 1 rle with old method : 6.127357482910156e-05 time for calcul the mask position with numpy : 0.008373498916625977 nb_pixel_total : 516 time to create 1 rle with old method : 0.0006213188171386719 time for calcul the mask position with numpy : 0.008336067199707031 nb_pixel_total : 2250 time to create 1 rle with old method : 0.0025899410247802734 time for calcul the mask position with numpy : 0.008389949798583984 nb_pixel_total : 386 time to create 1 rle with old method : 0.0004432201385498047 time for calcul the mask position with numpy : 0.008358240127563477 nb_pixel_total : 3004 time to create 1 rle with old method : 0.0034308433532714844 time for calcul the mask position with numpy : 0.008379459381103516 nb_pixel_total : 18 time to create 1 rle with old method : 4.100799560546875e-05 time for calcul the mask position with numpy : 0.00835728645324707 nb_pixel_total : 179 time to create 1 rle with old method : 0.00021219253540039062 time for calcul the mask position with numpy : 0.008363962173461914 nb_pixel_total : 499 time to create 1 rle with old method : 0.0005819797515869141 time for calcul the mask position with numpy : 0.008372306823730469 nb_pixel_total : 389 time to create 1 rle with old method : 0.0004527568817138672 time for calcul the mask position with numpy : 0.008391857147216797 nb_pixel_total : 107 time to create 1 rle with old method : 0.0001308917999267578 time for calcul the mask position with numpy : 0.008375167846679688 nb_pixel_total : 1174 time to create 1 rle with old method : 0.0013279914855957031 time for calcul the mask position with numpy : 0.008382558822631836 nb_pixel_total : 677 time to create 1 rle with old method : 0.0007658004760742188 time for calcul the mask position with numpy : 0.00838017463684082 nb_pixel_total : 111 time to create 1 rle with old method : 0.0001857280731201172 time for calcul the mask position with numpy : 0.008362293243408203 nb_pixel_total : 2033 time to create 1 rle with old method : 0.0022537708282470703 time for calcul the mask position with numpy : 0.008372068405151367 nb_pixel_total : 99 time to create 1 rle with old method : 0.0001251697540283203 time for calcul the mask position with numpy : 0.008420944213867188 nb_pixel_total : 149 time to create 1 rle with old method : 0.00017881393432617188 time for calcul the mask position with numpy : 0.008425712585449219 nb_pixel_total : 53 time to create 1 rle with old method : 6.937980651855469e-05 time for calcul the mask position with numpy : 0.00817418098449707 nb_pixel_total : 735 time to create 1 rle with old method : 0.0008554458618164062 time for calcul the mask position with numpy : 0.008142709732055664 nb_pixel_total : 304 time to create 1 rle with old method : 0.00036787986755371094 time for calcul the mask position with numpy : 0.008365631103515625 nb_pixel_total : 606 time to create 1 rle with old method : 0.0007224082946777344 time for calcul the mask position with numpy : 0.008362293243408203 nb_pixel_total : 238 time to create 1 rle with old method : 0.0002734661102294922 time for calcul the mask position with numpy : 0.008334159851074219 nb_pixel_total : 48 time to create 1 rle with old method : 8.392333984375e-05 time for calcul the mask position with numpy : 0.008039712905883789 nb_pixel_total : 968 time to create 1 rle with old method : 0.0010514259338378906 time for calcul the mask position with numpy : 0.008042573928833008 nb_pixel_total : 2364 time to create 1 rle with old method : 0.0026521682739257812 time for calcul the mask position with numpy : 0.00837254524230957 nb_pixel_total : 392 time to create 1 rle with old method : 0.0004684925079345703 time for calcul the mask position with numpy : 0.008423566818237305 nb_pixel_total : 31902 time to create 1 rle with old method : 0.03517436981201172 time for calcul the mask position with numpy : 0.008381128311157227 nb_pixel_total : 254 time to create 1 rle with old method : 0.000301361083984375 time for calcul the mask position with numpy : 0.00838923454284668 nb_pixel_total : 1042 time to create 1 rle with old method : 0.0012171268463134766 time for calcul the mask position with numpy : 0.008383512496948242 nb_pixel_total : 355 time to create 1 rle with old method : 0.0004165172576904297 time for calcul the mask position with numpy : 0.008357763290405273 nb_pixel_total : 330 time to create 1 rle with old method : 0.00038695335388183594 time for calcul the mask position with numpy : 0.008391141891479492 nb_pixel_total : 413 time to create 1 rle with old method : 0.00048828125 time for calcul the mask position with numpy : 0.008395910263061523 nb_pixel_total : 412 time to create 1 rle with old method : 0.0004916191101074219 time for calcul the mask position with numpy : 0.008370399475097656 nb_pixel_total : 1057 time to create 1 rle with old method : 0.001218557357788086 time for calcul the mask position with numpy : 0.008380413055419922 nb_pixel_total : 4 time to create 1 rle with old method : 1.6927719116210938e-05 time for calcul the mask position with numpy : 0.00834202766418457 nb_pixel_total : 2772 time to create 1 rle with old method : 0.0030775070190429688 time for calcul the mask position with numpy : 0.008390426635742188 nb_pixel_total : 497 time to create 1 rle with old method : 0.0005533695220947266 time for calcul the mask position with numpy : 0.008368492126464844 nb_pixel_total : 386 time to create 1 rle with old method : 0.00044798851013183594 time for calcul the mask position with numpy : 0.008348226547241211 nb_pixel_total : 810 time to create 1 rle with old method : 0.0009922981262207031 time for calcul the mask position with numpy : 0.008356809616088867 nb_pixel_total : 693 time to create 1 rle with old method : 0.0008339881896972656 time for calcul the mask position with numpy : 0.008348703384399414 nb_pixel_total : 351 time to create 1 rle with old method : 0.0003936290740966797 time for calcul the mask position with numpy : 0.008345603942871094 nb_pixel_total : 756 time to create 1 rle with old method : 0.0008647441864013672 time for calcul the mask position with numpy : 0.008389711380004883 nb_pixel_total : 1 time to create 1 rle with old method : 7.3909759521484375e-06 time for calcul the mask position with numpy : 0.008367300033569336 nb_pixel_total : 283 time to create 1 rle with old method : 0.0003333091735839844 time for calcul the mask position with numpy : 0.008364677429199219 nb_pixel_total : 78 time to create 1 rle with old method : 9.036064147949219e-05 time for calcul the mask position with numpy : 0.008405447006225586 nb_pixel_total : 18 time to create 1 rle with old method : 2.8848648071289062e-05 time for calcul the mask position with numpy : 0.0083160400390625 nb_pixel_total : 354 time to create 1 rle with old method : 0.00043487548828125 time for calcul the mask position with numpy : 0.008366107940673828 nb_pixel_total : 632 time to create 1 rle with old method : 0.0007395744323730469 time for calcul the mask position with numpy : 0.008331060409545898 nb_pixel_total : 713 time to create 1 rle with old method : 0.0008184909820556641 time for calcul the mask position with numpy : 0.008339405059814453 nb_pixel_total : 496 time to create 1 rle with old method : 0.0005810260772705078 time for calcul the mask position with numpy : 0.009334564208984375 nb_pixel_total : 205 time to create 1 rle with old method : 0.0002655982971191406 time for calcul the mask position with numpy : 0.008342504501342773 nb_pixel_total : 303 time to create 1 rle with old method : 0.0003559589385986328 time for calcul the mask position with numpy : 0.008360624313354492 nb_pixel_total : 273 time to create 1 rle with old method : 0.00036907196044921875 create new chi : 1.5879604816436768 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.005466938018798828 batch 1 Loaded 99 chid ids of type : 4230 Number RLEs to save : 9801 TO DO : save crop sub photo not yet done ! save time : 3.3464035987854004 nb_obj : 86 nb_hashtags : 7 time to prepare the origin masks : 1.941359281539917 time for calcul the mask position with numpy : 0.03338909149169922 nb_pixel_total : 1745938 time to create 1 rle with new method : 0.06287455558776855 time for calcul the mask position with numpy : 0.00750422477722168 nb_pixel_total : 67358 time to create 1 rle with old method : 0.07298493385314941 time for calcul the mask position with numpy : 0.006670236587524414 nb_pixel_total : 3253 time to create 1 rle with old method : 0.0036897659301757812 time for calcul the mask position with numpy : 0.006663322448730469 nb_pixel_total : 2387 time to create 1 rle with old method : 0.0028002262115478516 time for calcul the mask position with numpy : 0.006432056427001953 nb_pixel_total : 270 time to create 1 rle with old method : 0.0003440380096435547 time for calcul the mask position with numpy : 0.006005287170410156 nb_pixel_total : 605 time to create 1 rle with old method : 0.00072479248046875 time for calcul the mask position with numpy : 0.006061077117919922 nb_pixel_total : 283 time to create 1 rle with old method : 0.00035858154296875 time for calcul the mask position with numpy : 0.006136894226074219 nb_pixel_total : 38 time to create 1 rle with old method : 6.937980651855469e-05 time for calcul the mask position with numpy : 0.006201744079589844 nb_pixel_total : 7566 time to create 1 rle with old method : 0.008295059204101562 time for calcul the mask position with numpy : 0.006052732467651367 nb_pixel_total : 495 time to create 1 rle with old method : 0.0006453990936279297 time for calcul the mask position with numpy : 0.006233930587768555 nb_pixel_total : 1538 time to create 1 rle with old method : 0.0017902851104736328 time for calcul the mask position with numpy : 0.006391763687133789 nb_pixel_total : 1800 time to create 1 rle with old method : 0.0021224021911621094 time for calcul the mask position with numpy : 0.006091594696044922 nb_pixel_total : 125 time to create 1 rle with old method : 0.0001678466796875 time for calcul the mask position with numpy : 0.006011486053466797 nb_pixel_total : 1807 time to create 1 rle with old method : 0.002170085906982422 time for calcul the mask position with numpy : 0.006824016571044922 nb_pixel_total : 1500 time to create 1 rle with old method : 0.0017108917236328125 time for calcul the mask position with numpy : 0.006195068359375 nb_pixel_total : 502 time to create 1 rle with old method : 0.0006511211395263672 time for calcul the mask position with numpy : 0.005972146987915039 nb_pixel_total : 241 time to create 1 rle with old method : 0.0003299713134765625 time for calcul the mask position with numpy : 0.006003618240356445 nb_pixel_total : 4059 time to create 1 rle with old method : 0.004536867141723633 time for calcul the mask position with numpy : 0.005967855453491211 nb_pixel_total : 844 time to create 1 rle with old method : 0.0009639263153076172 time for calcul the mask position with numpy : 0.005983114242553711 nb_pixel_total : 191 time to create 1 rle with old method : 0.00023937225341796875 time for calcul the mask position with numpy : 0.0059735774993896484 nb_pixel_total : 1248 time to create 1 rle with old method : 0.0013675689697265625 time for calcul the mask position with numpy : 0.0059795379638671875 nb_pixel_total : 336 time to create 1 rle with old method : 0.00042366981506347656 time for calcul the mask position with numpy : 0.005980730056762695 nb_pixel_total : 1690 time to create 1 rle with old method : 0.001996755599975586 time for calcul the mask position with numpy : 0.006016254425048828 nb_pixel_total : 8627 time to create 1 rle with old method : 0.009813308715820312 time for calcul the mask position with numpy : 0.00686335563659668 nb_pixel_total : 33535 time to create 1 rle with old method : 0.03711223602294922 time for calcul the mask position with numpy : 0.005983829498291016 nb_pixel_total : 379 time to create 1 rle with old method : 0.0005276203155517578 time for calcul the mask position with numpy : 0.006115436553955078 nb_pixel_total : 1808 time to create 1 rle with old method : 0.002151966094970703 time for calcul the mask position with numpy : 0.006009101867675781 nb_pixel_total : 426 time to create 1 rle with old method : 0.0005660057067871094 time for calcul the mask position with numpy : 0.006091117858886719 nb_pixel_total : 572 time to create 1 rle with old method : 0.0007121562957763672 time for calcul the mask position with numpy : 0.006495475769042969 nb_pixel_total : 4014 time to create 1 rle with old method : 0.004779815673828125 time for calcul the mask position with numpy : 0.0087127685546875 nb_pixel_total : 230 time to create 1 rle with old method : 0.0002818107604980469 time for calcul the mask position with numpy : 0.010765790939331055 nb_pixel_total : 1837 time to create 1 rle with old method : 0.002148151397705078 time for calcul the mask position with numpy : 0.009981393814086914 nb_pixel_total : 528 time to create 1 rle with old method : 0.0007631778717041016 time for calcul the mask position with numpy : 0.009961128234863281 nb_pixel_total : 554 time to create 1 rle with old method : 0.0006973743438720703 time for calcul the mask position with numpy : 0.009925603866577148 nb_pixel_total : 604 time to create 1 rle with old method : 0.0007636547088623047 time for calcul the mask position with numpy : 0.009999275207519531 nb_pixel_total : 2411 time to create 1 rle with old method : 0.0027709007263183594 time for calcul the mask position with numpy : 0.009734869003295898 nb_pixel_total : 344 time to create 1 rle with old method : 0.000423431396484375 time for calcul the mask position with numpy : 0.010036468505859375 nb_pixel_total : 224 time to create 1 rle with old method : 0.0002827644348144531 time for calcul the mask position with numpy : 0.010138988494873047 nb_pixel_total : 1176 time to create 1 rle with old method : 0.0013821125030517578 time for calcul the mask position with numpy : 0.01019597053527832 nb_pixel_total : 4118 time to create 1 rle with old method : 0.004780292510986328 time for calcul the mask position with numpy : 0.010007143020629883 nb_pixel_total : 14 time to create 1 rle with old method : 5.245208740234375e-05 time for calcul the mask position with numpy : 0.009879827499389648 nb_pixel_total : 418 time to create 1 rle with old method : 0.0005137920379638672 time for calcul the mask position with numpy : 0.00992441177368164 nb_pixel_total : 182 time to create 1 rle with old method : 0.00022339820861816406 time for calcul the mask position with numpy : 0.010078668594360352 nb_pixel_total : 94 time to create 1 rle with old method : 0.00011873245239257812 time for calcul the mask position with numpy : 0.009978055953979492 nb_pixel_total : 1092 time to create 1 rle with old method : 0.0012769699096679688 time for calcul the mask position with numpy : 0.009072303771972656 nb_pixel_total : 177 time to create 1 rle with old method : 0.00022339820861816406 time for calcul the mask position with numpy : 0.005975008010864258 nb_pixel_total : 580 time to create 1 rle with old method : 0.0006692409515380859 time for calcul the mask position with numpy : 0.0059566497802734375 nb_pixel_total : 246 time to create 1 rle with old method : 0.0002892017364501953 time for calcul the mask position with numpy : 0.006931781768798828 nb_pixel_total : 105553 time to create 1 rle with old method : 0.11693072319030762 time for calcul the mask position with numpy : 0.006088972091674805 nb_pixel_total : 35 time to create 1 rle with old method : 7.128715515136719e-05 time for calcul the mask position with numpy : 0.0060236454010009766 nb_pixel_total : 2021 time to create 1 rle with old method : 0.0023703575134277344 time for calcul the mask position with numpy : 0.006044149398803711 nb_pixel_total : 55 time to create 1 rle with old method : 0.0001423358917236328 time for calcul the mask position with numpy : 0.006000995635986328 nb_pixel_total : 229 time to create 1 rle with old method : 0.00027561187744140625 time for calcul the mask position with numpy : 0.006006717681884766 nb_pixel_total : 264 time to create 1 rle with old method : 0.0003371238708496094 time for calcul the mask position with numpy : 0.005975246429443359 nb_pixel_total : 250 time to create 1 rle with old method : 0.00033211708068847656 time for calcul the mask position with numpy : 0.006102085113525391 nb_pixel_total : 620 time to create 1 rle with old method : 0.0007174015045166016 time for calcul the mask position with numpy : 0.00615239143371582 nb_pixel_total : 351 time to create 1 rle with old method : 0.0004303455352783203 time for calcul the mask position with numpy : 0.00613856315612793 nb_pixel_total : 270 time to create 1 rle with old method : 0.0003521442413330078 time for calcul the mask position with numpy : 0.006600141525268555 nb_pixel_total : 37026 time to create 1 rle with old method : 0.04045605659484863 time for calcul the mask position with numpy : 0.006185770034790039 nb_pixel_total : 1008 time to create 1 rle with old method : 0.0011930465698242188 time for calcul the mask position with numpy : 0.006021976470947266 nb_pixel_total : 350 time to create 1 rle with old method : 0.0004162788391113281 time for calcul the mask position with numpy : 0.006417512893676758 nb_pixel_total : 434 time to create 1 rle with old method : 0.0006704330444335938 time for calcul the mask position with numpy : 0.00724029541015625 nb_pixel_total : 502 time to create 1 rle with old method : 0.0007510185241699219 time for calcul the mask position with numpy : 0.006877422332763672 nb_pixel_total : 276 time to create 1 rle with old method : 0.000331878662109375 time for calcul the mask position with numpy : 0.006035327911376953 nb_pixel_total : 2255 time to create 1 rle with old method : 0.0025877952575683594 time for calcul the mask position with numpy : 0.0066144466400146484 nb_pixel_total : 508 time to create 1 rle with old method : 0.0006392002105712891 time for calcul the mask position with numpy : 0.0060291290283203125 nb_pixel_total : 880 time to create 1 rle with old method : 0.0010936260223388672 time for calcul the mask position with numpy : 0.006057262420654297 nb_pixel_total : 369 time to create 1 rle with old method : 0.0004601478576660156 time for calcul the mask position with numpy : 0.006044149398803711 nb_pixel_total : 3748 time to create 1 rle with old method : 0.0042438507080078125 time for calcul the mask position with numpy : 0.00606989860534668 nb_pixel_total : 779 time to create 1 rle with old method : 0.0009710788726806641 time for calcul the mask position with numpy : 0.006017446517944336 nb_pixel_total : 326 time to create 1 rle with old method : 0.0004067420959472656 time for calcul the mask position with numpy : 0.005986928939819336 nb_pixel_total : 458 time to create 1 rle with old method : 0.0005297660827636719 time for calcul the mask position with numpy : 0.00601506233215332 nb_pixel_total : 360 time to create 1 rle with old method : 0.00045990943908691406 time for calcul the mask position with numpy : 0.005990028381347656 nb_pixel_total : 901 time to create 1 rle with old method : 0.0011241436004638672 time for calcul the mask position with numpy : 0.006018638610839844 nb_pixel_total : 13 time to create 1 rle with old method : 6.508827209472656e-05 time for calcul the mask position with numpy : 0.0059947967529296875 nb_pixel_total : 369 time to create 1 rle with old method : 0.00041794776916503906 time for calcul the mask position with numpy : 0.006023883819580078 nb_pixel_total : 79 time to create 1 rle with old method : 0.00010967254638671875 time for calcul the mask position with numpy : 0.005957841873168945 nb_pixel_total : 223 time to create 1 rle with old method : 0.0002770423889160156 time for calcul the mask position with numpy : 0.006035804748535156 nb_pixel_total : 1132 time to create 1 rle with old method : 0.0013761520385742188 time for calcul the mask position with numpy : 0.0060117244720458984 nb_pixel_total : 421 time to create 1 rle with old method : 0.000507354736328125 time for calcul the mask position with numpy : 0.005998134613037109 nb_pixel_total : 642 time to create 1 rle with old method : 0.0007479190826416016 time for calcul the mask position with numpy : 0.006033658981323242 nb_pixel_total : 666 time to create 1 rle with old method : 0.0007803440093994141 time for calcul the mask position with numpy : 0.0060193538665771484 nb_pixel_total : 541 time to create 1 rle with old method : 0.0006635189056396484 time for calcul the mask position with numpy : 0.006409883499145508 nb_pixel_total : 205 time to create 1 rle with old method : 0.0003032684326171875 time for calcul the mask position with numpy : 0.006468296051025391 nb_pixel_total : 1019 time to create 1 rle with old method : 0.0012688636779785156 time for calcul the mask position with numpy : 0.006949901580810547 nb_pixel_total : 2 time to create 1 rle with old method : 3.4809112548828125e-05 time for calcul the mask position with numpy : 0.00650477409362793 nb_pixel_total : 196 time to create 1 rle with old method : 0.00024700164794921875 create new chi : 1.0594136714935303 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.0012390613555908203 batch 1 Loaded 87 chid ids of type : 4230 Number RLEs to save : 10274 TO DO : save crop sub photo not yet done ! save time : 0.6045680046081543 nb_obj : 94 nb_hashtags : 7 time to prepare the origin masks : 1.4122669696807861 time for calcul the mask position with numpy : 0.01786327362060547 nb_pixel_total : 1741605 time to create 1 rle with new method : 0.9880886077880859 time for calcul the mask position with numpy : 0.006815195083618164 nb_pixel_total : 62976 time to create 1 rle with old method : 0.07099199295043945 time for calcul the mask position with numpy : 0.006173610687255859 nb_pixel_total : 70 time to create 1 rle with old method : 0.00010514259338378906 time for calcul the mask position with numpy : 0.006439685821533203 nb_pixel_total : 973 time to create 1 rle with old method : 0.0011622905731201172 time for calcul the mask position with numpy : 0.006481170654296875 nb_pixel_total : 71 time to create 1 rle with old method : 9.369850158691406e-05 time for calcul the mask position with numpy : 0.006032228469848633 nb_pixel_total : 64 time to create 1 rle with old method : 8.821487426757812e-05 time for calcul the mask position with numpy : 0.0061070919036865234 nb_pixel_total : 55 time to create 1 rle with old method : 8.296966552734375e-05 time for calcul the mask position with numpy : 0.0062656402587890625 nb_pixel_total : 16 time to create 1 rle with old method : 6.079673767089844e-05 time for calcul the mask position with numpy : 0.0062406063079833984 nb_pixel_total : 4708 time to create 1 rle with old method : 0.0054128170013427734 time for calcul the mask position with numpy : 0.0066318511962890625 nb_pixel_total : 2406 time to create 1 rle with old method : 0.002773284912109375 time for calcul the mask position with numpy : 0.008310079574584961 nb_pixel_total : 4 time to create 1 rle with old method : 3.0040740966796875e-05 time for calcul the mask position with numpy : 0.0070841312408447266 nb_pixel_total : 229 time to create 1 rle with old method : 0.0002837181091308594 time for calcul the mask position with numpy : 0.00680088996887207 nb_pixel_total : 605 time to create 1 rle with old method : 0.0006885528564453125 time for calcul the mask position with numpy : 0.006123542785644531 nb_pixel_total : 563 time to create 1 rle with old method : 0.0006861686706542969 time for calcul the mask position with numpy : 0.006287574768066406 nb_pixel_total : 9880 time to create 1 rle with old method : 0.011305093765258789 time for calcul the mask position with numpy : 0.006212472915649414 nb_pixel_total : 519 time to create 1 rle with old method : 0.0006978511810302734 time for calcul the mask position with numpy : 0.00644993782043457 nb_pixel_total : 1486 time to create 1 rle with old method : 0.0016973018646240234 time for calcul the mask position with numpy : 0.006408214569091797 nb_pixel_total : 50 time to create 1 rle with old method : 0.00010180473327636719 time for calcul the mask position with numpy : 0.006359577178955078 nb_pixel_total : 3522 time to create 1 rle with old method : 0.00422978401184082 time for calcul the mask position with numpy : 0.006757497787475586 nb_pixel_total : 162 time to create 1 rle with old method : 0.00022459030151367188 time for calcul the mask position with numpy : 0.006540060043334961 nb_pixel_total : 1835 time to create 1 rle with old method : 0.0022873878479003906 time for calcul the mask position with numpy : 0.007546186447143555 nb_pixel_total : 106 time to create 1 rle with old method : 0.0002257823944091797 time for calcul the mask position with numpy : 0.0063970088958740234 nb_pixel_total : 170 time to create 1 rle with old method : 0.00028824806213378906 time for calcul the mask position with numpy : 0.00626826286315918 nb_pixel_total : 1790 time to create 1 rle with old method : 0.002238750457763672 time for calcul the mask position with numpy : 0.006833314895629883 nb_pixel_total : 490 time to create 1 rle with old method : 0.0006089210510253906 time for calcul the mask position with numpy : 0.007112264633178711 nb_pixel_total : 1667 time to create 1 rle with old method : 0.0019860267639160156 time for calcul the mask position with numpy : 0.0067729949951171875 nb_pixel_total : 1783 time to create 1 rle with old method : 0.0022618770599365234 time for calcul the mask position with numpy : 0.007097959518432617 nb_pixel_total : 234 time to create 1 rle with old method : 0.00037670135498046875 time for calcul the mask position with numpy : 0.007580757141113281 nb_pixel_total : 3590 time to create 1 rle with old method : 0.004255533218383789 time for calcul the mask position with numpy : 0.0068514347076416016 nb_pixel_total : 348 time to create 1 rle with old method : 0.0004897117614746094 time for calcul the mask position with numpy : 0.006518125534057617 nb_pixel_total : 795 time to create 1 rle with old method : 0.0009505748748779297 time for calcul the mask position with numpy : 0.00657343864440918 nb_pixel_total : 380 time to create 1 rle with old method : 0.0004527568817138672 time for calcul the mask position with numpy : 0.0062122344970703125 nb_pixel_total : 197 time to create 1 rle with old method : 0.0002694129943847656 time for calcul the mask position with numpy : 0.006426572799682617 nb_pixel_total : 1275 time to create 1 rle with old method : 0.0015227794647216797 time for calcul the mask position with numpy : 0.006223201751708984 nb_pixel_total : 986 time to create 1 rle with old method : 0.0011951923370361328 time for calcul the mask position with numpy : 0.005983591079711914 nb_pixel_total : 2002 time to create 1 rle with old method : 0.0023169517517089844 time for calcul the mask position with numpy : 0.006834506988525391 nb_pixel_total : 218 time to create 1 rle with old method : 0.0002789497375488281 time for calcul the mask position with numpy : 0.006840705871582031 nb_pixel_total : 38954 time to create 1 rle with old method : 0.04552006721496582 time for calcul the mask position with numpy : 0.006211042404174805 nb_pixel_total : 1703 time to create 1 rle with old method : 0.0019905567169189453 time for calcul the mask position with numpy : 0.0070037841796875 nb_pixel_total : 540 time to create 1 rle with old method : 0.0006537437438964844 time for calcul the mask position with numpy : 0.006720066070556641 nb_pixel_total : 2476 time to create 1 rle with old method : 0.002788066864013672 time for calcul the mask position with numpy : 0.008704423904418945 nb_pixel_total : 769 time to create 1 rle with old method : 0.0009684562683105469 time for calcul the mask position with numpy : 0.007176637649536133 nb_pixel_total : 347 time to create 1 rle with old method : 0.0004458427429199219 time for calcul the mask position with numpy : 0.00682830810546875 nb_pixel_total : 4007 time to create 1 rle with old method : 0.0044667720794677734 time for calcul the mask position with numpy : 0.006173372268676758 nb_pixel_total : 156 time to create 1 rle with old method : 0.00021719932556152344 time for calcul the mask position with numpy : 0.0063703060150146484 nb_pixel_total : 1484 time to create 1 rle with old method : 0.0018172264099121094 time for calcul the mask position with numpy : 0.0063402652740478516 nb_pixel_total : 2438 time to create 1 rle with old method : 0.0028662681579589844 time for calcul the mask position with numpy : 0.006720781326293945 nb_pixel_total : 226 time to create 1 rle with old method : 0.0002894401550292969 time for calcul the mask position with numpy : 0.006288290023803711 nb_pixel_total : 529 time to create 1 rle with old method : 0.0006396770477294922 time for calcul the mask position with numpy : 0.00630640983581543 nb_pixel_total : 46 time to create 1 rle with old method : 9.274482727050781e-05 time for calcul the mask position with numpy : 0.00606989860534668 nb_pixel_total : 1330 time to create 1 rle with old method : 0.0015413761138916016 time for calcul the mask position with numpy : 0.00641632080078125 nb_pixel_total : 392 time to create 1 rle with old method : 0.0004756450653076172 time for calcul the mask position with numpy : 0.0060999393463134766 nb_pixel_total : 396 time to create 1 rle with old method : 0.00046563148498535156 time for calcul the mask position with numpy : 0.0062901973724365234 nb_pixel_total : 2854 time to create 1 rle with old method : 0.003321409225463867 time for calcul the mask position with numpy : 0.00635981559753418 nb_pixel_total : 35 time to create 1 rle with old method : 7.224082946777344e-05 time for calcul the mask position with numpy : 0.006331682205200195 nb_pixel_total : 186 time to create 1 rle with old method : 0.00022339820861816406 time for calcul the mask position with numpy : 0.006307840347290039 nb_pixel_total : 1967 time to create 1 rle with old method : 0.002309083938598633 time for calcul the mask position with numpy : 0.006354808807373047 nb_pixel_total : 108 time to create 1 rle with old method : 0.00014090538024902344 time for calcul the mask position with numpy : 0.006028890609741211 nb_pixel_total : 150 time to create 1 rle with old method : 0.0002167224884033203 time for calcul the mask position with numpy : 0.006128787994384766 nb_pixel_total : 754 time to create 1 rle with old method : 0.0008771419525146484 time for calcul the mask position with numpy : 0.006196022033691406 nb_pixel_total : 1191 time to create 1 rle with old method : 0.0013837814331054688 time for calcul the mask position with numpy : 0.007264852523803711 nb_pixel_total : 105529 time to create 1 rle with old method : 0.1171114444732666 time for calcul the mask position with numpy : 0.00654149055480957 nb_pixel_total : 28 time to create 1 rle with old method : 9.679794311523438e-05 time for calcul the mask position with numpy : 0.006655454635620117 nb_pixel_total : 1956 time to create 1 rle with old method : 0.0023195743560791016 time for calcul the mask position with numpy : 0.0064275264739990234 nb_pixel_total : 2937 time to create 1 rle with old method : 0.003573179244995117 time for calcul the mask position with numpy : 0.006610870361328125 nb_pixel_total : 179 time to create 1 rle with old method : 0.0002295970916748047 time for calcul the mask position with numpy : 0.006850004196166992 nb_pixel_total : 396 time to create 1 rle with old method : 0.0005006790161132812 time for calcul the mask position with numpy : 0.006329774856567383 nb_pixel_total : 214 time to create 1 rle with old method : 0.00026798248291015625 time for calcul the mask position with numpy : 0.006230592727661133 nb_pixel_total : 2520 time to create 1 rle with old method : 0.0029790401458740234 time for calcul the mask position with numpy : 0.0062220096588134766 nb_pixel_total : 99 time to create 1 rle with old method : 0.00013256072998046875 time for calcul the mask position with numpy : 0.006140232086181641 nb_pixel_total : 866 time to create 1 rle with old method : 0.001071929931640625 time for calcul the mask position with numpy : 0.006316661834716797 nb_pixel_total : 395 time to create 1 rle with old method : 0.0004665851593017578 time for calcul the mask position with numpy : 0.006094694137573242 nb_pixel_total : 257 time to create 1 rle with old method : 0.0002930164337158203 time for calcul the mask position with numpy : 0.0063114166259765625 nb_pixel_total : 37363 time to create 1 rle with old method : 0.04137110710144043 time for calcul the mask position with numpy : 0.006384372711181641 nb_pixel_total : 1045 time to create 1 rle with old method : 0.0012860298156738281 time for calcul the mask position with numpy : 0.009921789169311523 nb_pixel_total : 326 time to create 1 rle with old method : 0.00040435791015625 time for calcul the mask position with numpy : 0.009923458099365234 nb_pixel_total : 380 time to create 1 rle with old method : 0.00047278404235839844 time for calcul the mask position with numpy : 0.009857177734375 nb_pixel_total : 450 time to create 1 rle with old method : 0.0005593299865722656 time for calcul the mask position with numpy : 0.009919881820678711 nb_pixel_total : 265 time to create 1 rle with old method : 0.0003428459167480469 time for calcul the mask position with numpy : 0.010274887084960938 nb_pixel_total : 1889 time to create 1 rle with old method : 0.0023114681243896484 time for calcul the mask position with numpy : 0.011587858200073242 nb_pixel_total : 478 time to create 1 rle with old method : 0.0005850791931152344 time for calcul the mask position with numpy : 0.010826349258422852 nb_pixel_total : 355 time to create 1 rle with old method : 0.00044846534729003906 time for calcul the mask position with numpy : 0.00994729995727539 nb_pixel_total : 495 time to create 1 rle with old method : 0.0006167888641357422 time for calcul the mask position with numpy : 0.009911298751831055 nb_pixel_total : 515 time to create 1 rle with old method : 0.0006418228149414062 time for calcul the mask position with numpy : 0.009828329086303711 nb_pixel_total : 805 time to create 1 rle with old method : 0.0009672641754150391 time for calcul the mask position with numpy : 0.0097503662109375 nb_pixel_total : 455 time to create 1 rle with old method : 0.000514984130859375 time for calcul the mask position with numpy : 0.009973287582397461 nb_pixel_total : 275 time to create 1 rle with old method : 0.0003476142883300781 time for calcul the mask position with numpy : 0.01043391227722168 nb_pixel_total : 84 time to create 1 rle with old method : 0.0001270771026611328 time for calcul the mask position with numpy : 0.010451078414916992 nb_pixel_total : 20 time to create 1 rle with old method : 3.9577484130859375e-05 time for calcul the mask position with numpy : 0.010884761810302734 nb_pixel_total : 210 time to create 1 rle with old method : 0.0003657341003417969 time for calcul the mask position with numpy : 0.010419845581054688 nb_pixel_total : 371 time to create 1 rle with old method : 0.0004532337188720703 time for calcul the mask position with numpy : 0.006877899169921875 nb_pixel_total : 674 time to create 1 rle with old method : 0.0008084774017333984 time for calcul the mask position with numpy : 0.006244659423828125 nb_pixel_total : 462 time to create 1 rle with old method : 0.0005254745483398438 time for calcul the mask position with numpy : 0.006432771682739258 nb_pixel_total : 191 time to create 1 rle with old method : 0.0002472400665283203 time for calcul the mask position with numpy : 0.006238698959350586 nb_pixel_total : 248 time to create 1 rle with old method : 0.00030684471130371094 create new chi : 2.069448471069336 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.0012845993041992188 batch 1 Loaded 95 chid ids of type : 4230 Number RLEs to save : 10448 TO DO : save crop sub photo not yet done ! save time : 2.4752445220947266 map_output_result : {1337909766: (0.0, 'Should be the crop_list due to order', 0.0), 1337909751: (0.0, 'Should be the crop_list due to order', 0.0), 1337909750: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1337909766, 1337909751, 1337909750] Looping around the photos to save general results len do output : 3 /1337909766.Didn't retrieve data . /1337909751.Didn't retrieve data . /1337909750.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2591081') ('4234', None, '1337909766', None, None, None, None, None, '2591081') ('4234', None, None, None, None, None, None, None, '2591081') ('4234', None, '1337909751', None, None, None, None, None, '2591081') ('4234', None, None, None, None, None, None, None, '2591081') ('4234', None, '1337909750', None, None, None, None, None, '2591081') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.03957724571228027 save_final save missing photos in datou_result : time spend for datou_step_exec : 16.054633855819702 time spend to save output : 0.039907217025756836 total time spend for step 4 : 16.09454107284546 step5:crop_condition Sat Feb 15 19:21:15 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4230 Loading chi in step crop for list_pids : 3 ! batch 1 Loaded 281 chid ids of type : 4230 begin to crop the class : papier param for this class : {'min_score': 0.6} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 110 About to insert : list_path_to_insert length 110 new photo from crops ! About to upload 110 photos upload in portfolio : 4869462 init cache_photo without model_param we have 110 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739643678_1861816 we have uploaded 110 photos in the portfolio 4869462 time of upload the photos Elapsed time : 35.97634506225586 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.6} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 38 About to insert : list_path_to_insert length 38 new photo from crops ! About to upload 38 photos upload in portfolio : 4869462 init cache_photo without model_param we have 38 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739643715_1861816 we have uploaded 38 photos in the portfolio 4869462 time of upload the photos Elapsed time : 9.639572620391846 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.6} filtre for class : metal hashtag_id of this class : 492628673 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739643724_1861816 we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.7080497741699219 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.6} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 15 About to insert : list_path_to_insert length 15 new photo from crops ! About to upload 15 photos upload in portfolio : 4869462 init cache_photo without model_param we have 15 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739643726_1861816 we have uploaded 15 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.6123881340026855 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.6} filtre for class : autre hashtag_id of this class : 494826614 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 31 About to insert : list_path_to_insert length 31 new photo from crops ! About to upload 31 photos upload in portfolio : 4869462 init cache_photo without model_param we have 31 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739643730_1861816 we have uploaded 31 photos in the portfolio 4869462 time of upload the photos Elapsed time : 7.141516923904419 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.6} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.6} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1337909766, 1337909751, 1337909750] Looping around the photos to save general results len do output : 195 /1337948003Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948004Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948006Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948007Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948008Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948009Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948010Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948011Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948012Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948014Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948015Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948016Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948017Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948018Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948019Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948020Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948021Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948022Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948023Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948024Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948025Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948026Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948027Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948028Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948029Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948030Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948031Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948032Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948033Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948034Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948035Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948036Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948037Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948038Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948039Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948040Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948041Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948042Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948043Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948044Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948045Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948046Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948048Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948049Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948050Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948051Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948052Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948053Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948054Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948055Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948056Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948057Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948058Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948059Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948060Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948061Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948062Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948063Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948064Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948065Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948066Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948067Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948068Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948069Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948070Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948071Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948072Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948073Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948074Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948075Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948076Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948077Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948078Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948079Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948080Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948081Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948082Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948083Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948084Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948085Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948086Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948087Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948088Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948089Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948090Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948091Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948093Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948094Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948095Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948096Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948097Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948098Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948099Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948100Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948101Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948102Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948103Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948104Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948105Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948106Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948107Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948108Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948109Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948110Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948111Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948112Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948113Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948114Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948115Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948116Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948119Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948120Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948121Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948122Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948123Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948124Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948125Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948126Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948127Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948128Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948129Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948130Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948131Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948132Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948133Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948134Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948135Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948136Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948137Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948138Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948139Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948140Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948141Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948142Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948143Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948144Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948145Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948146Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948147Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948148Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948149Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948150Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948151Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948152Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948153Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948154Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948155Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948156Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948157Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948158Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948159Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948160Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948161Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948162Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948163Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948164Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948165Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948166Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948167Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948168Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948169Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948170Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948171Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948172Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948173Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948174Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948175Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948176Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948177Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948178Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948179Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948180Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948181Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948182Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948183Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948184Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948185Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948186Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948187Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948188Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948189Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948190Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948191Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948192Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948193Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948194Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948195Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948196Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948197Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948198Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948199Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948200Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948201Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948202Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337948203Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2591081') ('4234', None, '1337909766', None, None, None, None, None, '2591081') ('4234', None, None, None, None, None, None, None, '2591081') ('4234', None, '1337909751', None, None, None, None, None, '2591081') ('4234', None, None, None, None, None, None, None, '2591081') ('4234', None, '1337909750', None, None, None, None, None, '2591081') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 588 time used for this insertion : 0.03785443305969238 save_final save missing photos in datou_result : time spend for datou_step_exec : 61.457425355911255 time spend to save output : 0.045352935791015625 total time spend for step 5 : 61.50277829170227 step6:thcl Sat Feb 15 19:22:17 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! we are using the classfication for only one thcl 3237 time to import caffe and check if the image exist : 0.010766267776489258 time to convert the images to numpy array : 0.05109572410583496 time to import caffe and check if the image exist : 0.013468265533447266 time to convert the images to numpy array : 0.04776954650878906 time to import caffe and check if the image exist : 0.013240814208984375 time to convert the images to numpy array : 0.05255770683288574 time to import caffe and check if the image exist : 0.017688274383544922 time to convert the images to numpy array : 0.05007481575012207 time to import caffe and check if the image exist : 0.019242525100708008 time to convert the images to numpy array : 0.05310487747192383 time to import caffe and check if the image exist : 0.023028135299682617 time to convert the images to numpy array : 0.05089426040649414 time to import caffe and check if the image exist : 0.013622045516967773 time to convert the images to numpy array : 0.060898780822753906 time to import caffe and check if the image exist : 0.009665250778198242 time to convert the images to numpy array : 0.06422090530395508 time to import caffe and check if the image exist : 0.01646900177001953 time to convert the images to numpy array : 0.059661865234375 time to import caffe and check if the image exist : 0.030031919479370117 time to convert the images to numpy array : 0.04679131507873535 total time to convert the images to numpy array : 0.3308568000793457 list photo_ids error: [] list photo_ids correct : [1337948003, 1337948004, 1337948006, 1337948007, 1337948008, 1337948009, 1337948010, 1337948011, 1337948012, 1337948014, 1337948015, 1337948016, 1337948017, 1337948018, 1337948019, 1337948020, 1337948021, 1337948022, 1337948023, 1337948024, 1337948129, 1337948130, 1337948131, 1337948132, 1337948133, 1337948134, 1337948135, 1337948136, 1337948137, 1337948138, 1337948139, 1337948140, 1337948141, 1337948142, 1337948143, 1337948144, 1337948145, 1337948146, 1337948147, 1337948148, 1337948149, 1337948150, 1337948151, 1337948152, 1337948153, 1337948154, 1337948155, 1337948156, 1337948157, 1337948158, 1337948159, 1337948160, 1337948161, 1337948162, 1337948163, 1337948164, 1337948165, 1337948166, 1337948167, 1337948168, 1337948045, 1337948046, 1337948048, 1337948049, 1337948050, 1337948051, 1337948052, 1337948053, 1337948054, 1337948055, 1337948056, 1337948057, 1337948058, 1337948059, 1337948060, 1337948061, 1337948062, 1337948063, 1337948064, 1337948065, 1337948107, 1337948108, 1337948109, 1337948110, 1337948111, 1337948112, 1337948113, 1337948114, 1337948115, 1337948116, 1337948119, 1337948120, 1337948121, 1337948122, 1337948123, 1337948124, 1337948125, 1337948126, 1337948127, 1337948128, 1337948086, 1337948087, 1337948088, 1337948089, 1337948090, 1337948091, 1337948093, 1337948094, 1337948095, 1337948096, 1337948097, 1337948098, 1337948099, 1337948100, 1337948101, 1337948102, 1337948103, 1337948104, 1337948105, 1337948106, 1337948025, 1337948026, 1337948027, 1337948028, 1337948029, 1337948030, 1337948031, 1337948032, 1337948033, 1337948034, 1337948035, 1337948036, 1337948037, 1337948038, 1337948039, 1337948040, 1337948041, 1337948042, 1337948043, 1337948044, 1337948189, 1337948190, 1337948191, 1337948192, 1337948193, 1337948194, 1337948195, 1337948196, 1337948197, 1337948198, 1337948199, 1337948200, 1337948201, 1337948202, 1337948203, 1337948066, 1337948067, 1337948068, 1337948069, 1337948070, 1337948071, 1337948072, 1337948073, 1337948074, 1337948075, 1337948076, 1337948077, 1337948078, 1337948079, 1337948080, 1337948081, 1337948082, 1337948083, 1337948084, 1337948085, 1337948169, 1337948170, 1337948171, 1337948172, 1337948173, 1337948174, 1337948175, 1337948176, 1337948177, 1337948178, 1337948179, 1337948180, 1337948181, 1337948182, 1337948183, 1337948184, 1337948185, 1337948186, 1337948187, 1337948188] number of photos to traite : 195 try to delete the photos incorrect in DB tagging for thcl : 3237 To do loadFromThcl(), then load ParamDescType : thcl3237 thcls : [{'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'}] thcl {'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'} Update svm_hashtag_type_desc : 5561 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) To loadFromThcl() : net_5561 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10770 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_rubbia_refus_2500 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_rubbia_refus_2500 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_rubbia_refus_2500 /data/models_weight/learn_rubbia_refus_2500/caffemodel size_local : 94358479 size in s3 : 94358479 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:16 caffemodel already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt size_local : 32544 size in s3 : 32544 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:15 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:09:52 mean.npy already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/synset_words.txt size_local : 334 size in s3 : 334 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:10:06 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt caffemodel_filename : /data/models_weight/learn_rubbia_refus_2500/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10551 max_wait_temp : 1 max_wait : 0 dict_keys(['prob', 'pool5']) time used to do the prepocess of the images : 0.9043190479278564 time used to do the prediction : 0.744098424911499 save descriptor for thcl : 3237 time to traite the descriptors : 1.0997729301452637 Catched exception ! Connect or reconnect ! storage_type for insertDescriptorsMulti : 3 To insert : 1337948003 To insert : 1337948004 To insert : 1337948006 To insert : 1337948007 To insert : 1337948008 To insert : 1337948009 To insert : 1337948010 To insert : 1337948011 To insert : 1337948012 To insert : 1337948014 To insert : 1337948015 To insert : 1337948016 To insert : 1337948017 To insert : 1337948018 To insert : 1337948019 To insert : 1337948020 To insert : 1337948021 To insert : 1337948022 To insert : 1337948023 To insert : 1337948024 To insert : 1337948129 To insert : 1337948130 To insert : 1337948131 To insert : 1337948132 To insert : 1337948133 To insert : 1337948134 To insert : 1337948135 To insert : 1337948136 To insert : 1337948137 To insert : 1337948138 To insert : 1337948139 To insert : 1337948140 To insert : 1337948141 To insert : 1337948142 To insert : 1337948143 To insert : 1337948144 To insert : 1337948145 To insert : 1337948146 To insert : 1337948147 To insert : 1337948148 To insert : 1337948149 To insert : 1337948150 To insert : 1337948151 To insert : 1337948152 To insert : 1337948153 To insert : 1337948154 To insert : 1337948155 To insert : 1337948156 To insert : 1337948157 To insert : 1337948158 To insert : 1337948159 To insert : 1337948160 To insert : 1337948161 To insert : 1337948162 To insert : 1337948163 To insert : 1337948164 To insert : 1337948165 To insert : 1337948166 To insert : 1337948167 To insert : 1337948168 To insert : 1337948045 To insert : 1337948046 To insert : 1337948048 To insert : 1337948049 To insert : 1337948050 To insert : 1337948051 To insert : 1337948052 To insert : 1337948053 To insert : 1337948054 To insert : 1337948055 To insert : 1337948056 To insert : 1337948057 To insert : 1337948058 To insert : 1337948059 To insert : 1337948060 To insert : 1337948061 To insert : 1337948062 To insert : 1337948063 To insert : 1337948064 To insert : 1337948065 To insert : 1337948107 To insert : 1337948108 To insert : 1337948109 To insert : 1337948110 To insert : 1337948111 To insert : 1337948112 To insert : 1337948113 To insert : 1337948114 To insert : 1337948115 To insert : 1337948116 To insert : 1337948119 To insert : 1337948120 To insert : 1337948121 To insert : 1337948122 To insert : 1337948123 To insert : 1337948124 To insert : 1337948125 To insert : 1337948126 To insert : 1337948127 To insert : 1337948128 To insert : 1337948086 To insert : 1337948087 To insert : 1337948088 To insert : 1337948089 To insert : 1337948090 To insert : 1337948091 To insert : 1337948093 To insert : 1337948094 To insert : 1337948095 To insert : 1337948096 To insert : 1337948097 To insert : 1337948098 To insert : 1337948099 To insert : 1337948100 To insert : 1337948101 To insert : 1337948102 To insert : 1337948103 To insert : 1337948104 To insert : 1337948105 To insert : 1337948106 To insert : 1337948025 To insert : 1337948026 To insert : 1337948027 To insert : 1337948028 To insert : 1337948029 To insert : 1337948030 To insert : 1337948031 To insert : 1337948032 To insert : 1337948033 To insert : 1337948034 To insert : 1337948035 To insert : 1337948036 To insert : 1337948037 To insert : 1337948038 To insert : 1337948039 To insert : 1337948040 To insert : 1337948041 To insert : 1337948042 To insert : 1337948043 To insert : 1337948044 To insert : 1337948189 To insert : 1337948190 To insert : 1337948191 To insert : 1337948192 To insert : 1337948193 To insert : 1337948194 To insert : 1337948195 To insert : 1337948196 To insert : 1337948197 To insert : 1337948198 To insert : 1337948199 To insert : 1337948200 To insert : 1337948201 To insert : 1337948202 To insert : 1337948203 To insert : 1337948066 To insert : 1337948067 To insert : 1337948068 To insert : 1337948069 To insert : 1337948070 To insert : 1337948071 To insert : 1337948072 To insert : 1337948073 To insert : 1337948074 To insert : 1337948075 To insert : 1337948076 To insert : 1337948077 To insert : 1337948078 To insert : 1337948079 To insert : 1337948080 To insert : 1337948081 To insert : 1337948082 To insert : 1337948083 To insert : 1337948084 To insert : 1337948085 To insert : 1337948169 To insert : 1337948170 To insert : 1337948171 To insert : 1337948172 To insert : 1337948173 To insert : 1337948174 To insert : 1337948175 To insert : 1337948176 To insert : 1337948177 To insert : 1337948178 To insert : 1337948179 To insert : 1337948180 To insert : 1337948181 To insert : 1337948182 To insert : 1337948183 To insert : 1337948184 To insert : 1337948185 To insert : 1337948186 To insert : 1337948187 To insert : 1337948188 time to insert the descriptors : 50.35279154777527 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos SAVE THCL : begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 300 time used for this insertion : 0.027084827423095703 save missing photos in datou_result : time spend for datou_step_exec : 57.40811562538147 time spend to save output : 0.08271408081054688 total time spend for step 6 : 57.49082970619202 step7:ventilate_hashtags_in_portfolio Sat Feb 15 19:23:14 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 20587832 get user id for portfolio 20587832 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20587832 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_fonce','PET_clair','Tetrapak','carton','Carton','PEHD','papier','Film_plastique','environnement','metal','pet_clair','Papier','flou','refus','autre','pehd','mal_croppe','PET_fonce')) AND mptpi.`min_score`=0.6 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20587832 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_fonce','PET_clair','Tetrapak','carton','Carton','PEHD','papier','Film_plastique','environnement','metal','pet_clair','Papier','flou','refus','autre','pehd','mal_croppe','PET_fonce')) AND mptpi.`min_score`=0.6 To do lien utilise dans velours : https://www.fotonower.com/velours/20588765,20588760,20588775,20588762,20588773,20588764,20588766,20588767,20588772,20588769,20588776,20588771,20588774?tags=papier,tetrapak,carton,metal,pehd,refus,flou,environnement,pet_clair,autre,pet_fonce,mal_croppe,film_plastique&datou_id_consolidate=4235&port_consolidate=20587832 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1337909766, 1337909751, 1337909750] Looping around the photos to save general results len do output : 1 /20587832. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2591081') ('4234', None, '1337909766', None, None, None, None, None, '2591081') ('4234', None, None, None, None, None, None, None, '2591081') ('4234', None, '1337909751', None, None, None, None, None, '2591081') ('4234', None, None, None, None, None, None, None, '2591081') ('4234', None, '1337909750', None, None, None, None, None, '2591081') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 4 time used for this insertion : 0.012729644775390625 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.7039852142333984 time spend to save output : 0.013070821762084961 total time spend for step 7 : 1.7170560359954834 step8:final Sat Feb 15 19:23:16 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1337909766: ('0.22290592518625865',), 1337909751: ('0.22290592518625865',), 1337909750: ('0.22290592518625865',)} new output for save of step final : {1337909766: ('0.22290592518625865',), 1337909751: ('0.22290592518625865',), 1337909750: ('0.22290592518625865',)} [1337909766, 1337909751, 1337909750] Looping around the photos to save general results len do output : 3 /1337909766.Didn't retrieve data . /1337909751.Didn't retrieve data . /1337909750.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2591081') ('4234', None, '1337909766', None, None, None, None, None, '2591081') ('4234', None, None, None, None, None, None, None, '2591081') ('4234', None, '1337909751', None, None, None, None, None, '2591081') ('4234', None, None, None, None, None, None, None, '2591081') ('4234', None, '1337909750', None, None, None, None, None, '2591081') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.011637449264526367 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.13110971450805664 time spend to save output : 0.011905670166015625 total time spend for step 8 : 0.14301538467407227 step9:velours_tree Sat Feb 15 19:23:16 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure list_portfolios : 20588765,20588760,20588775,20588762,20588773,20588764,20588766,20588767,20588772,20588769,20588776,20588771,20588774 photo desc type : 5561 - Retrieving photos to tag... query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20588765 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20588760 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20588775 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20588762 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20588773 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20588764 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20588766 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20588767 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20588772 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20588769 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20588776 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20588771 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20588774 ORDER BY ph.size desc - Loading descriptors... Size : 2048 len(descriptors) : 2643 Compute structured hierarchical clustering... ward : AgglomerativeClustering(n_clusters=6) ward.labels_ : [0 4 1 ... 3 2 2] Elapsed time: 4.386608123779297 graph_id used : 78952 - Beta version, working pretty good on 11-5-16 ! https://www.fotonower.com/velours/20588765,20588760,20588775,20588762,20588773,20588764,20588766,20588767,20588772,20588769,20588776,20588771,20588774?tags=papier,tetrapak,carton,metal,pehd,refus,flou,environnement,pet_clair,autre,pet_fonce,mal_croppe,film_plastique&datou_id_consolidate=4235&port_consolidate=20587832&tree_id=78952 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : velours_tree we use saveGeneral [1337909766, 1337909751, 1337909750] Looping around the photos to save general results len do output : 1 /20587832Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2591081') ('4234', None, '1337909766', None, None, None, None, None, '2591081') ('4234', None, None, None, None, None, None, None, '2591081') ('4234', None, '1337909751', None, None, None, None, None, '2591081') ('4234', None, None, None, None, None, None, None, '2591081') ('4234', None, '1337909750', None, None, None, None, None, '2591081') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 4 time used for this insertion : 0.01309967041015625 save_final save missing photos in datou_result : time spend for datou_step_exec : 212.73850774765015 time spend to save output : 0.019640445709228516 total time spend for step 9 : 212.75814819335938 step10:send_mail_cod Sat Feb 15 19:26:49 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Qualipapia_P20587832_15-02-2025_19_26_49.pdf 20588747 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette205887471739644009 20588742 imagette205887421739644010 20588757 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette205887571739644010 20588744 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette205887441739644012 20588755 imagette205887551739644013 20588746 imagette205887461739644013 20588748 imagette205887481739644013 20588754 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette205887541739644013 20588751 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette205887511739644014 20588758 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette205887581739644016 20588753 imagette205887531739644016 20588756 imagette205887561739644016 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20587832 and hashtag_type = 4230 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20588765,20588760,20588775,20588762,20588773,20588764,20588766,20588767,20588772,20588769,20588776,20588771,20588774?tags=papier,tetrapak,carton,metal,pehd,refus,flou,environnement,pet_clair,autre,pet_fonce,mal_croppe,film_plastique&datou_id_consolidate=4235&port_consolidate=20587832&tree_id=78952 your option no_mail is active, we will not send the real mail to your client args[1337909766] : ((1337909766, -7.121482389800247, 492609224), (1337909766, -0.4017062483724828, 496442774), '0.22290592518625865') apple ((1337909766, -7.121482389800247, 492609224), (1337909766, -0.4017062483724828, 496442774), '0.22290592518625865') We are sending mail with results at cod@fotonower.com args[1337909751] : ((1337909751, -7.098568046193153, 492609224), (1337909751, -0.42599197301070163, 496442774), '0.22290592518625865') apple ((1337909751, -7.098568046193153, 492609224), (1337909751, -0.42599197301070163, 496442774), '0.22290592518625865') We are sending mail with results at cod@fotonower.com args[1337909750] : ((1337909750, -7.108969347317072, 492609224), (1337909750, -0.39254610274566765, 496442774), '0.22290592518625865') apple ((1337909750, -7.108969347317072, 492609224), (1337909750, -0.39254610274566765, 496442774), '0.22290592518625865') We are sending mail with results at cod@fotonower.com refus_total : 0.22290592518625865 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20587832 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20587832_15-02-2025_19_26_49.pdf results_Qualipapia_P20587832_15-02-2025_19_26_49.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20587832_15-02-2025_19_26_49.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4234','20587832','results_Qualipapia_P20587832_15-02-2025_19_26_49.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20587832_15-02-2025_19_26_49.pdf','pdf','','0.19','0.22290592518625865') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1337909766, 1337909751, 1337909750] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2591081') ('4234', None, '1337909766', None, None, None, None, None, '2591081') ('4234', None, None, None, None, None, None, None, '2591081') ('4234', None, '1337909751', None, None, None, None, None, '2591081') ('4234', None, None, None, None, None, None, None, '2591081') ('4234', None, '1337909750', None, None, None, None, None, '2591081') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.017099380493164062 save_final save missing photos in datou_result : time spend for datou_step_exec : 8.060900688171387 time spend to save output : 0.017338037490844727 total time spend for step 10 : 8.078238725662231 step11:split_time_score Sat Feb 15 19:26:57 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 3442, 'mtr_user_id': 31, 'name': 'classifieur_2camions_valcor_021122_v1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'deux_camions,camion_droite,camion_gauche,pas_de_camion', 'svm_portfolios_learning': '7659379,7659034,7657685,7657114', 'photo_hashtag_type': 4458, 'photo_desc_type': 5723, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107760533,2107760534,2107760535,2107760536'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('15', 45),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 15022025 20587832 Nombre de photos uploadées : 45 / 23040 (0%) 15022025 20587832 Nombre de photos taguées (types de déchets): 45 / 45 (100%) 15022025 20587832 Nombre de photos taguées (volume) : 0 / 45 (0%) elapsed_time : load_data_split_time_score 1.6689300537109375e-06 elapsed_time : order_list_meta_photo_and_scores 5.9604644775390625e-06 ????????????????????????????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0017445087432861328 elapsed_time : insert_dashboard_record_day_entry 0.024631977081298828 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.19915031927009344 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20587589_15-02-2025_16_37_20.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20587589 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20587589 AND mptpi.`type`=4230 To do Qualite : 0.2413886034835304 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20587590_15-02-2025_16_43_43.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20587590 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20587590 AND mptpi.`type`=4230 To do Qualite : 0.24208895078954654 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20587591_15-02-2025_15_16_41.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20587591 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20587591 AND mptpi.`type`=4230 To do Qualite : 0.2264213403816684 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20587592_15-02-2025_15_31_42.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20587592 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20587592 AND mptpi.`type`=4230 To do Qualite : 0.22063934371361915 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20587593_15-02-2025_15_02_31.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20587593 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20587593 AND mptpi.`type`=4230 To do Qualite : 0.23614686311479277 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20587594_15-02-2025_15_10_21.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20587594 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20587594 AND mptpi.`type`=4230 To do Qualite : 0.18742928698095865 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20587829_15-02-2025_17_02_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20587829 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20587829 AND mptpi.`type`=4230 To do Qualite : 0.21155241687847937 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20587830_15-02-2025_17_09_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20587830 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20587830 AND mptpi.`type`=4230 To do Qualite : 0.23185081615812542 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20587831_15-02-2025_17_50_38.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20587831 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20587831 AND mptpi.`type`=4230 To do Qualite : 0.20720850843255065 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20587832_15-02-2025_19_26_49.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20587832 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20587832 AND mptpi.`type`=4230 To do Qualite : 0.19132518832464887 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20587833_15-02-2025_15_34_39.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20587833 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20587833 AND mptpi.`type`=4230 To do Qualite : 0.21976665625005454 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20588020_15-02-2025_16_02_38.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20588020 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20588020 AND mptpi.`type`=4230 To do Qualite : 0.23151356792258032 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20588021_15-02-2025_16_33_53.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20588021 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20588021 AND mptpi.`type`=4230 To do Qualite : 0.2621999938673687 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20588022_15-02-2025_16_22_14.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20588022 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20588022 AND mptpi.`type`=4230 To do Qualite : 0.24246208844052905 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20588023_15-02-2025_16_11_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20588023 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20588023 AND mptpi.`type`=4230 To do Qualite : 0.23837118901784263 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20588024_15-02-2025_16_16_56.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20588024 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20588024 AND mptpi.`type`=4230 To do Qualite : 0.17866166047138649 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20588294_15-02-2025_16_34_27.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20588294 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20588294 AND mptpi.`type`=4230 To do Qualite : 0.23769750805307543 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20588493_15-02-2025_18_17_42.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20588493 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20588493 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20588494 order by id desc limit 1 Qualite : 0.24309681907837521 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20588495_15-02-2025_18_28_20.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20588495 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20588495 AND mptpi.`type`=4230 To do Qualite : 0.25263644906751326 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20588496_15-02-2025_18_33_25.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20588496 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20588496 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20588497 order by id desc limit 1 Qualite : 0.24335039866199737 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20588498_15-02-2025_17_51_26.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20588498 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20588498 AND mptpi.`type`=4230 To do Qualite : 0.23511030006247688 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20588499_15-02-2025_17_33_02.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20588499 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20588499 AND mptpi.`type`=4230 To do Qualite : 0.2375143045751391 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20588500_15-02-2025_18_31_44.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20588500 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20588500 AND mptpi.`type`=4230 To do Qualite : 0.27131940368734186 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20588501_15-02-2025_17_11_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20588501 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20588501 AND mptpi.`type`=4230 To do Qualite : 0.23938951279138712 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20588502_15-02-2025_17_16_40.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20588502 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20588502 AND mptpi.`type`=4230 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'15022025': {'nb_upload': 45, 'nb_taggue_class': 45, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1337909766, 1337909751, 1337909750] Looping around the photos to save general results len do output : 1 /20587832Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2591081') ('4234', None, '1337909766', None, None, None, None, None, '2591081') ('4234', None, None, None, None, None, None, None, '2591081') ('4234', None, '1337909751', None, None, None, None, None, '2591081') ('4234', None, None, None, None, None, None, None, '2591081') ('4234', None, '1337909750', None, None, None, None, None, '2591081') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 4 time used for this insertion : 0.10101485252380371 save_final save missing photos in datou_result : time spend for datou_step_exec : 3.4349522590637207 time spend to save output : 0.10143113136291504 total time spend for step 11 : 3.5363833904266357 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 3 set_done_treatment 54.94user 28.55system 6:36.51elapsed 21%CPU (0avgtext+0avgdata 2524768maxresident)k 260504inputs+35720outputs (293major+2425963minor)pagefaults 0swaps