python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 4234 -l 5 ' -s datou_current_4234 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1179035 load datou : 4234 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load THCL from format json or kwargs add thcl : 3237 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig add pdt : 5561 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4234, datou_cur_ids : ['2537864'] with mtr_portfolio_ids : ['20143824'] and first list_photo_ids : [] new path : /proc/1179035/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, brightness, blur_detection, rle_unique_nms_with_priority, crop_condition, thcl, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , BFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 2 ; length of list_pids : 2 ; length of list_args : 2 time to download the photos : 0.2733135223388672 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Sat Feb 1 18:20:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 5685 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-01 18:20:32.507652: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-01 18:20:32.539538: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493020000 Hz 2025-02-01 18:20:32.541702: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fe428000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-01 18:20:32.541759: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-01 18:20:32.545735: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-01 18:20:32.806124: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2d70bfb0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-01 18:20:32.806185: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-01 18:20:32.807445: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-01 18:20:32.808120: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-01 18:20:32.812436: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-01 18:20:32.831682: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-01 18:20:32.833249: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-01 18:20:32.863970: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-01 18:20:32.868757: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-01 18:20:32.923463: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-01 18:20:32.925046: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-01 18:20:32.925552: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-01 18:20:32.926659: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-01 18:20:32.926680: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-01 18:20:32.926692: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-01 18:20:32.928404: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 5201 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-01 18:20:33.361108: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-01 18:20:33.361185: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-01 18:20:33.361202: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-01 18:20:33.361218: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-01 18:20:33.361233: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-01 18:20:33.361247: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-01 18:20:33.361261: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-01 18:20:33.361276: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-01 18:20:33.362305: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-01 18:20:33.363466: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-01 18:20:33.363527: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-01 18:20:33.363543: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-01 18:20:33.363558: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-01 18:20:33.363572: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-01 18:20:33.363587: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-01 18:20:33.363601: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-01 18:20:33.363615: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-01 18:20:33.364638: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-01 18:20:33.364669: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-01 18:20:33.364678: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-01 18:20:33.364686: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-01 18:20:33.365762: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 5201 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-01 18:20:43.343069: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-01 18:20:43.698586: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 2 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -118.91094 max: 147.78750 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 7.939338684082031e-05 nb_pixel_total : 460 time to create 1 rle with old method : 0.0009517669677734375 length of segment : 27 time for calcul the mask position with numpy : 4.601478576660156e-05 nb_pixel_total : 111 time to create 1 rle with old method : 0.0002372264862060547 length of segment : 28 time for calcul the mask position with numpy : 4.267692565917969e-05 nb_pixel_total : 74 time to create 1 rle with old method : 0.0001785755157470703 length of segment : 20 time for calcul the mask position with numpy : 5.7697296142578125e-05 nb_pixel_total : 948 time to create 1 rle with old method : 0.0018069744110107422 length of segment : 36 time for calcul the mask position with numpy : 0.008092403411865234 nb_pixel_total : 14999 time to create 1 rle with old method : 0.023883819580078125 length of segment : 153 time for calcul the mask position with numpy : 4.9591064453125e-05 nb_pixel_total : 799 time to create 1 rle with old method : 0.0010576248168945312 length of segment : 32 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -118.92266 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 13 time for calcul the mask position with numpy : 6.198883056640625e-05 nb_pixel_total : 56 time to create 1 rle with old method : 0.00013303756713867188 length of segment : 10 time for calcul the mask position with numpy : 4.458427429199219e-05 nb_pixel_total : 281 time to create 1 rle with old method : 0.0005285739898681641 length of segment : 15 time for calcul the mask position with numpy : 4.7206878662109375e-05 nb_pixel_total : 890 time to create 1 rle with old method : 0.0013432502746582031 length of segment : 34 time for calcul the mask position with numpy : 3.337860107421875e-05 nb_pixel_total : 30 time to create 1 rle with old method : 7.2479248046875e-05 length of segment : 6 time for calcul the mask position with numpy : 4.2438507080078125e-05 nb_pixel_total : 190 time to create 1 rle with old method : 0.0003294944763183594 length of segment : 27 time for calcul the mask position with numpy : 0.00010538101196289062 nb_pixel_total : 908 time to create 1 rle with old method : 0.0013031959533691406 length of segment : 32 time for calcul the mask position with numpy : 4.649162292480469e-05 nb_pixel_total : 318 time to create 1 rle with old method : 0.0005388259887695312 length of segment : 16 time for calcul the mask position with numpy : 3.790855407714844e-05 nb_pixel_total : 270 time to create 1 rle with old method : 0.0005078315734863281 length of segment : 29 time for calcul the mask position with numpy : 9.322166442871094e-05 nb_pixel_total : 3444 time to create 1 rle with old method : 0.004423856735229492 length of segment : 65 time for calcul the mask position with numpy : 5.7697296142578125e-05 nb_pixel_total : 367 time to create 1 rle with old method : 0.0007481575012207031 length of segment : 17 time for calcul the mask position with numpy : 3.409385681152344e-05 nb_pixel_total : 34 time to create 1 rle with old method : 8.58306884765625e-05 length of segment : 7 time for calcul the mask position with numpy : 5.7697296142578125e-05 nb_pixel_total : 816 time to create 1 rle with old method : 0.0011756420135498047 length of segment : 45 time for calcul the mask position with numpy : 5.078315734863281e-05 nb_pixel_total : 808 time to create 1 rle with old method : 0.0010609626770019531 length of segment : 50 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -120.39922 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 0.00017118453979492188 nb_pixel_total : 6101 time to create 1 rle with old method : 0.007952690124511719 length of segment : 120 time for calcul the mask position with numpy : 9.5367431640625e-05 nb_pixel_total : 1228 time to create 1 rle with old method : 0.002783060073852539 length of segment : 59 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -107.69609 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 10 time for calcul the mask position with numpy : 0.00017023086547851562 nb_pixel_total : 8021 time to create 1 rle with old method : 0.013326644897460938 length of segment : 131 time for calcul the mask position with numpy : 3.552436828613281e-05 nb_pixel_total : 231 time to create 1 rle with old method : 0.0003275871276855469 length of segment : 17 time for calcul the mask position with numpy : 4.5299530029296875e-05 nb_pixel_total : 996 time to create 1 rle with old method : 0.0012171268463134766 length of segment : 56 time for calcul the mask position with numpy : 5.698204040527344e-05 nb_pixel_total : 1581 time to create 1 rle with old method : 0.0020787715911865234 length of segment : 29 time for calcul the mask position with numpy : 4.935264587402344e-05 nb_pixel_total : 1301 time to create 1 rle with old method : 0.00176239013671875 length of segment : 27 time for calcul the mask position with numpy : 3.170967102050781e-05 nb_pixel_total : 81 time to create 1 rle with old method : 0.00013327598571777344 length of segment : 17 time for calcul the mask position with numpy : 3.5762786865234375e-05 nb_pixel_total : 337 time to create 1 rle with old method : 0.0005156993865966797 length of segment : 20 time for calcul the mask position with numpy : 5.7220458984375e-05 nb_pixel_total : 1764 time to create 1 rle with old method : 0.002144336700439453 length of segment : 53 time for calcul the mask position with numpy : 3.5762786865234375e-05 nb_pixel_total : 388 time to create 1 rle with old method : 0.0004942417144775391 length of segment : 23 time for calcul the mask position with numpy : 5.3882598876953125e-05 nb_pixel_total : 1789 time to create 1 rle with old method : 0.002139568328857422 length of segment : 52 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 146.89297 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 6.0558319091796875e-05 nb_pixel_total : 1587 time to create 1 rle with old method : 0.0019648075103759766 length of segment : 69 time for calcul the mask position with numpy : 3.9577484130859375e-05 nb_pixel_total : 448 time to create 1 rle with old method : 0.0006742477416992188 length of segment : 17 time for calcul the mask position with numpy : 5.3882598876953125e-05 nb_pixel_total : 1390 time to create 1 rle with old method : 0.0015954971313476562 length of segment : 134 Processing 1 images image shape: (400, 400, 3) min: 8.00000 max: 202.00000 molded_images shape: (1, 640, 640, 3) min: -93.19219 max: 68.12969 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -118.08672 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 7.843971252441406e-05 nb_pixel_total : 1004 time to create 1 rle with old method : 0.0022079944610595703 length of segment : 53 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 301 time to create 1 rle with old method : 0.0005893707275390625 length of segment : 27 time for calcul the mask position with numpy : 0.0001277923583984375 nb_pixel_total : 1249 time to create 1 rle with old method : 0.0026574134826660156 length of segment : 46 time for calcul the mask position with numpy : 0.0003268718719482422 nb_pixel_total : 10986 time to create 1 rle with old method : 0.016425132751464844 length of segment : 182 time for calcul the mask position with numpy : 3.3855438232421875e-05 nb_pixel_total : 170 time to create 1 rle with old method : 0.0002720355987548828 length of segment : 17 time for calcul the mask position with numpy : 3.3855438232421875e-05 nb_pixel_total : 289 time to create 1 rle with old method : 0.0003821849822998047 length of segment : 26 time for calcul the mask position with numpy : 3.0040740966796875e-05 nb_pixel_total : 153 time to create 1 rle with old method : 0.00022673606872558594 length of segment : 15 time for calcul the mask position with numpy : 3.24249267578125e-05 nb_pixel_total : 253 time to create 1 rle with old method : 0.0003681182861328125 length of segment : 21 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -109.77422 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 4.3392181396484375e-05 nb_pixel_total : 137 time to create 1 rle with old method : 0.0002722740173339844 length of segment : 13 time for calcul the mask position with numpy : 3.123283386230469e-05 nb_pixel_total : 153 time to create 1 rle with old method : 0.00022339820861816406 length of segment : 17 time for calcul the mask position with numpy : 3.1948089599609375e-05 nb_pixel_total : 115 time to create 1 rle with old method : 0.00018525123596191406 length of segment : 11 time for calcul the mask position with numpy : 3.62396240234375e-05 nb_pixel_total : 525 time to create 1 rle with old method : 0.0006754398345947266 length of segment : 30 time for calcul the mask position with numpy : 4.291534423828125e-05 nb_pixel_total : 805 time to create 1 rle with old method : 0.0010175704956054688 length of segment : 75 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -121.37578 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 8.58306884765625e-05 nb_pixel_total : 663 time to create 1 rle with old method : 0.0013427734375 length of segment : 46 time for calcul the mask position with numpy : 6.341934204101562e-05 nb_pixel_total : 162 time to create 1 rle with old method : 0.0004458427429199219 length of segment : 18 time for calcul the mask position with numpy : 0.00011730194091796875 nb_pixel_total : 1951 time to create 1 rle with old method : 0.003037691116333008 length of segment : 54 time for calcul the mask position with numpy : 4.982948303222656e-05 nb_pixel_total : 450 time to create 1 rle with old method : 0.0005884170532226562 length of segment : 33 time for calcul the mask position with numpy : 7.557868957519531e-05 nb_pixel_total : 2470 time to create 1 rle with old method : 0.0029952526092529297 length of segment : 41 time for calcul the mask position with numpy : 5.91278076171875e-05 nb_pixel_total : 1283 time to create 1 rle with old method : 0.0015718936920166016 length of segment : 39 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 141.57656 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 9.107589721679688e-05 nb_pixel_total : 1723 time to create 1 rle with old method : 0.0029418468475341797 length of segment : 87 time for calcul the mask position with numpy : 5.316734313964844e-05 nb_pixel_total : 347 time to create 1 rle with old method : 0.0006377696990966797 length of segment : 31 time for calcul the mask position with numpy : 6.747245788574219e-05 nb_pixel_total : 732 time to create 1 rle with old method : 0.0013799667358398438 length of segment : 50 time for calcul the mask position with numpy : 0.0002434253692626953 nb_pixel_total : 1888 time to create 1 rle with old method : 0.003093719482421875 length of segment : 52 time for calcul the mask position with numpy : 4.172325134277344e-05 nb_pixel_total : 490 time to create 1 rle with old method : 0.0006811618804931641 length of segment : 28 time for calcul the mask position with numpy : 0.00016880035400390625 nb_pixel_total : 2571 time to create 1 rle with old method : 0.0030469894409179688 length of segment : 171 Processing 1 images image shape: (280, 400, 3) min: 7.00000 max: 190.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 55.40469 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0013325214385986328 nb_pixel_total : 106548 time to create 1 rle with old method : 0.12172722816467285 length of segment : 281 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 148.33437 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 6.079673767089844e-05 nb_pixel_total : 1580 time to create 1 rle with old method : 0.0018625259399414062 length of segment : 88 time for calcul the mask position with numpy : 3.5762786865234375e-05 nb_pixel_total : 287 time to create 1 rle with old method : 0.00039577484130859375 length of segment : 24 time for calcul the mask position with numpy : 3.0279159545898438e-05 nb_pixel_total : 74 time to create 1 rle with old method : 0.00011968612670898438 length of segment : 15 time for calcul the mask position with numpy : 3.62396240234375e-05 nb_pixel_total : 383 time to create 1 rle with old method : 0.0004801750183105469 length of segment : 44 time for calcul the mask position with numpy : 4.38690185546875e-05 nb_pixel_total : 711 time to create 1 rle with old method : 0.0009405612945556641 length of segment : 57 time for calcul the mask position with numpy : 3.0994415283203125e-05 nb_pixel_total : 79 time to create 1 rle with old method : 0.00013399124145507812 length of segment : 9 time for calcul the mask position with numpy : 4.267692565917969e-05 nb_pixel_total : 781 time to create 1 rle with old method : 0.0009837150573730469 length of segment : 34 time for calcul the mask position with numpy : 3.337860107421875e-05 nb_pixel_total : 235 time to create 1 rle with old method : 0.0003287792205810547 length of segment : 19 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.02187 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 38 time for calcul the mask position with numpy : 6.246566772460938e-05 nb_pixel_total : 489 time to create 1 rle with old method : 0.0007205009460449219 length of segment : 28 time for calcul the mask position with numpy : 3.3855438232421875e-05 nb_pixel_total : 153 time to create 1 rle with old method : 0.0002429485321044922 length of segment : 17 time for calcul the mask position with numpy : 3.2901763916015625e-05 nb_pixel_total : 237 time to create 1 rle with old method : 0.00035834312438964844 length of segment : 14 time for calcul the mask position with numpy : 5.412101745605469e-05 nb_pixel_total : 1279 time to create 1 rle with old method : 0.0015099048614501953 length of segment : 49 time for calcul the mask position with numpy : 3.1948089599609375e-05 nb_pixel_total : 179 time to create 1 rle with old method : 0.0002636909484863281 length of segment : 16 time for calcul the mask position with numpy : 3.266334533691406e-05 nb_pixel_total : 250 time to create 1 rle with old method : 0.00036334991455078125 length of segment : 19 time for calcul the mask position with numpy : 3.24249267578125e-05 nb_pixel_total : 317 time to create 1 rle with old method : 0.0004482269287109375 length of segment : 16 time for calcul the mask position with numpy : 3.170967102050781e-05 nb_pixel_total : 197 time to create 1 rle with old method : 0.00029468536376953125 length of segment : 16 time for calcul the mask position with numpy : 3.337860107421875e-05 nb_pixel_total : 214 time to create 1 rle with old method : 0.0003414154052734375 length of segment : 24 time for calcul the mask position with numpy : 3.170967102050781e-05 nb_pixel_total : 144 time to create 1 rle with old method : 0.00021147727966308594 length of segment : 18 time for calcul the mask position with numpy : 3.337860107421875e-05 nb_pixel_total : 296 time to create 1 rle with old method : 0.00041604042053222656 length of segment : 19 time for calcul the mask position with numpy : 2.9087066650390625e-05 nb_pixel_total : 79 time to create 1 rle with old method : 0.00013756752014160156 length of segment : 13 time for calcul the mask position with numpy : 0.000102996826171875 nb_pixel_total : 2267 time to create 1 rle with old method : 0.0026116371154785156 length of segment : 124 time for calcul the mask position with numpy : 3.218650817871094e-05 nb_pixel_total : 97 time to create 1 rle with old method : 0.00015926361083984375 length of segment : 11 time for calcul the mask position with numpy : 3.0517578125e-05 nb_pixel_total : 110 time to create 1 rle with old method : 0.00019812583923339844 length of segment : 13 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 834 time to create 1 rle with old method : 0.0010089874267578125 length of segment : 40 time for calcul the mask position with numpy : 3.9577484130859375e-05 nb_pixel_total : 260 time to create 1 rle with old method : 0.0003731250762939453 length of segment : 21 time for calcul the mask position with numpy : 4.267692565917969e-05 nb_pixel_total : 696 time to create 1 rle with old method : 0.0008673667907714844 length of segment : 35 time for calcul the mask position with numpy : 3.218650817871094e-05 nb_pixel_total : 147 time to create 1 rle with old method : 0.0002205371856689453 length of segment : 15 time for calcul the mask position with numpy : 2.9802322387695312e-05 nb_pixel_total : 73 time to create 1 rle with old method : 0.0001251697540283203 length of segment : 12 time for calcul the mask position with numpy : 3.24249267578125e-05 nb_pixel_total : 193 time to create 1 rle with old method : 0.0002834796905517578 length of segment : 15 time for calcul the mask position with numpy : 3.0279159545898438e-05 nb_pixel_total : 105 time to create 1 rle with old method : 0.0001628398895263672 length of segment : 14 time for calcul the mask position with numpy : 2.8371810913085938e-05 nb_pixel_total : 48 time to create 1 rle with old method : 8.96453857421875e-05 length of segment : 11 time for calcul the mask position with numpy : 3.2901763916015625e-05 nb_pixel_total : 244 time to create 1 rle with old method : 0.0003464221954345703 length of segment : 16 time for calcul the mask position with numpy : 3.743171691894531e-05 nb_pixel_total : 428 time to create 1 rle with old method : 0.0005710124969482422 length of segment : 24 time for calcul the mask position with numpy : 3.6716461181640625e-05 nb_pixel_total : 387 time to create 1 rle with old method : 0.000492095947265625 length of segment : 28 time for calcul the mask position with numpy : 5.53131103515625e-05 nb_pixel_total : 1364 time to create 1 rle with old method : 0.0018208026885986328 length of segment : 21 time for calcul the mask position with numpy : 6.318092346191406e-05 nb_pixel_total : 990 time to create 1 rle with old method : 0.0015645027160644531 length of segment : 47 time for calcul the mask position with numpy : 0.0001239776611328125 nb_pixel_total : 2566 time to create 1 rle with old method : 0.003049135208129883 length of segment : 145 time for calcul the mask position with numpy : 3.457069396972656e-05 nb_pixel_total : 231 time to create 1 rle with old method : 0.0002994537353515625 length of segment : 21 time for calcul the mask position with numpy : 7.104873657226562e-05 nb_pixel_total : 1667 time to create 1 rle with old method : 0.002906322479248047 length of segment : 41 time for calcul the mask position with numpy : 4.744529724121094e-05 nb_pixel_total : 269 time to create 1 rle with old method : 0.0005433559417724609 length of segment : 18 time for calcul the mask position with numpy : 4.363059997558594e-05 nb_pixel_total : 200 time to create 1 rle with old method : 0.0003933906555175781 length of segment : 16 time for calcul the mask position with numpy : 6.914138793945312e-05 nb_pixel_total : 798 time to create 1 rle with old method : 0.0014438629150390625 length of segment : 35 time for calcul the mask position with numpy : 6.747245788574219e-05 nb_pixel_total : 1313 time to create 1 rle with old method : 0.0024194717407226562 length of segment : 19 time for calcul the mask position with numpy : 4.458427429199219e-05 nb_pixel_total : 279 time to create 1 rle with old method : 0.0005629062652587891 length of segment : 19 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 317 time to create 1 rle with old method : 0.00060272216796875 length of segment : 23 time for calcul the mask position with numpy : 0.00012993812561035156 nb_pixel_total : 2356 time to create 1 rle with old method : 0.00397491455078125 length of segment : 128 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 145.98281 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 5.078315734863281e-05 nb_pixel_total : 614 time to create 1 rle with old method : 0.0007855892181396484 length of segment : 31 time for calcul the mask position with numpy : 4.2438507080078125e-05 nb_pixel_total : 846 time to create 1 rle with old method : 0.0011746883392333984 length of segment : 17 time for calcul the mask position with numpy : 5.030632019042969e-05 nb_pixel_total : 1277 time to create 1 rle with old method : 0.0015063285827636719 length of segment : 49 time for calcul the mask position with numpy : 3.552436828613281e-05 nb_pixel_total : 400 time to create 1 rle with old method : 0.0005340576171875 length of segment : 34 time for calcul the mask position with numpy : 7.104873657226562e-05 nb_pixel_total : 668 time to create 1 rle with old method : 0.0008678436279296875 length of segment : 52 time for calcul the mask position with numpy : 0.0005104541778564453 nb_pixel_total : 30757 time to create 1 rle with old method : 0.035227298736572266 length of segment : 292 time for calcul the mask position with numpy : 5.7697296142578125e-05 nb_pixel_total : 110 time to create 1 rle with old method : 0.00028586387634277344 length of segment : 24 time for calcul the mask position with numpy : 0.00011157989501953125 nb_pixel_total : 1547 time to create 1 rle with old method : 0.0018870830535888672 length of segment : 112 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 133.60000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0005502700805664062 nb_pixel_total : 40950 time to create 1 rle with old method : 0.0447850227355957 length of segment : 294 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -118.03594 max: 147.78750 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 4.029273986816406e-05 nb_pixel_total : 412 time to create 1 rle with old method : 0.0005440711975097656 length of segment : 36 time for calcul the mask position with numpy : 4.3392181396484375e-05 nb_pixel_total : 914 time to create 1 rle with old method : 0.001194000244140625 length of segment : 39 time for calcul the mask position with numpy : 3.123283386230469e-05 nb_pixel_total : 95 time to create 1 rle with old method : 0.00014209747314453125 length of segment : 27 time for calcul the mask position with numpy : 2.9087066650390625e-05 nb_pixel_total : 62 time to create 1 rle with old method : 0.00010395050048828125 length of segment : 20 time for calcul the mask position with numpy : 4.291534423828125e-05 nb_pixel_total : 800 time to create 1 rle with old method : 0.0010557174682617188 length of segment : 37 time for calcul the mask position with numpy : 4.0531158447265625e-05 nb_pixel_total : 509 time to create 1 rle with old method : 0.0007381439208984375 length of segment : 38 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -118.00078 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 3.9577484130859375e-05 nb_pixel_total : 358 time to create 1 rle with old method : 0.00051116943359375 length of segment : 17 time for calcul the mask position with numpy : 4.076957702636719e-05 nb_pixel_total : 667 time to create 1 rle with old method : 0.0008902549743652344 length of segment : 54 time for calcul the mask position with numpy : 3.3855438232421875e-05 nb_pixel_total : 330 time to create 1 rle with old method : 0.00048470497131347656 length of segment : 15 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 1048 time to create 1 rle with old method : 0.001379251480102539 length of segment : 35 time for calcul the mask position with numpy : 5.3882598876953125e-05 nb_pixel_total : 1007 time to create 1 rle with old method : 0.0015463829040527344 length of segment : 46 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -120.90703 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 0.0001404285430908203 nb_pixel_total : 6799 time to create 1 rle with old method : 0.00786280632019043 length of segment : 109 time for calcul the mask position with numpy : 5.3882598876953125e-05 nb_pixel_total : 1052 time to create 1 rle with old method : 0.001485586166381836 length of segment : 16 time for calcul the mask position with numpy : 8.034706115722656e-05 nb_pixel_total : 955 time to create 1 rle with old method : 0.001432657241821289 length of segment : 106 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -112.01641 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 0.00017642974853515625 nb_pixel_total : 8180 time to create 1 rle with old method : 0.014551401138305664 length of segment : 136 time for calcul the mask position with numpy : 5.555152893066406e-05 nb_pixel_total : 236 time to create 1 rle with old method : 0.0003445148468017578 length of segment : 18 time for calcul the mask position with numpy : 5.6743621826171875e-05 nb_pixel_total : 1041 time to create 1 rle with old method : 0.0013039112091064453 length of segment : 57 time for calcul the mask position with numpy : 4.267692565917969e-05 nb_pixel_total : 349 time to create 1 rle with old method : 0.0005283355712890625 length of segment : 20 time for calcul the mask position with numpy : 3.910064697265625e-05 nb_pixel_total : 150 time to create 1 rle with old method : 0.0002453327178955078 length of segment : 11 time for calcul the mask position with numpy : 4.029273986816406e-05 nb_pixel_total : 147 time to create 1 rle with old method : 0.00027632713317871094 length of segment : 19 time for calcul the mask position with numpy : 3.409385681152344e-05 nb_pixel_total : 59 time to create 1 rle with old method : 0.00010657310485839844 length of segment : 14 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 147.52578 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 6.937980651855469e-05 nb_pixel_total : 472 time to create 1 rle with old method : 0.0011036396026611328 length of segment : 17 time for calcul the mask position with numpy : 6.103515625e-05 nb_pixel_total : 91 time to create 1 rle with old method : 0.00028443336486816406 length of segment : 11 time for calcul the mask position with numpy : 7.152557373046875e-05 nb_pixel_total : 659 time to create 1 rle with old method : 0.0014162063598632812 length of segment : 44 time for calcul the mask position with numpy : 0.00011372566223144531 nb_pixel_total : 1321 time to create 1 rle with old method : 0.0025358200073242188 length of segment : 115 time for calcul the mask position with numpy : 6.818771362304688e-05 nb_pixel_total : 535 time to create 1 rle with old method : 0.0012066364288330078 length of segment : 21 Processing 1 images image shape: (400, 400, 3) min: 5.00000 max: 202.00000 molded_images shape: (1, 640, 640, 3) min: -95.34453 max: 67.11016 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -113.29766 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 11 time for calcul the mask position with numpy : 4.482269287109375e-05 nb_pixel_total : 313 time to create 1 rle with old method : 0.0004413127899169922 length of segment : 28 time for calcul the mask position with numpy : 4.220008850097656e-05 nb_pixel_total : 834 time to create 1 rle with old method : 0.0010883808135986328 length of segment : 33 time for calcul the mask position with numpy : 3.7670135498046875e-05 nb_pixel_total : 513 time to create 1 rle with old method : 0.0007126331329345703 length of segment : 18 time for calcul the mask position with numpy : 3.981590270996094e-05 nb_pixel_total : 571 time to create 1 rle with old method : 0.0007424354553222656 length of segment : 37 time for calcul the mask position with numpy : 3.0279159545898438e-05 nb_pixel_total : 112 time to create 1 rle with old method : 0.00025534629821777344 length of segment : 12 time for calcul the mask position with numpy : 4.9114227294921875e-05 nb_pixel_total : 1233 time to create 1 rle with old method : 0.0015251636505126953 length of segment : 45 time for calcul the mask position with numpy : 0.00023603439331054688 nb_pixel_total : 10864 time to create 1 rle with old method : 0.012892484664916992 length of segment : 153 time for calcul the mask position with numpy : 5.0067901611328125e-05 nb_pixel_total : 376 time to create 1 rle with old method : 0.0005197525024414062 length of segment : 27 time for calcul the mask position with numpy : 4.601478576660156e-05 nb_pixel_total : 459 time to create 1 rle with old method : 0.0006663799285888672 length of segment : 26 time for calcul the mask position with numpy : 3.147125244140625e-05 nb_pixel_total : 82 time to create 1 rle with old method : 0.0001385211944580078 length of segment : 11 time for calcul the mask position with numpy : 3.0517578125e-05 nb_pixel_total : 97 time to create 1 rle with old method : 0.00016808509826660156 length of segment : 9 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -114.24688 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 3.838539123535156e-05 nb_pixel_total : 126 time to create 1 rle with old method : 0.0002071857452392578 length of segment : 12 time for calcul the mask position with numpy : 3.743171691894531e-05 nb_pixel_total : 443 time to create 1 rle with old method : 0.0006039142608642578 length of segment : 20 time for calcul the mask position with numpy : 4.100799560546875e-05 nb_pixel_total : 810 time to create 1 rle with old method : 0.0010454654693603516 length of segment : 38 time for calcul the mask position with numpy : 3.147125244140625e-05 nb_pixel_total : 141 time to create 1 rle with old method : 0.00021076202392578125 length of segment : 17 time for calcul the mask position with numpy : 2.9087066650390625e-05 nb_pixel_total : 24 time to create 1 rle with old method : 8.702278137207031e-05 length of segment : 4 time for calcul the mask position with numpy : 2.956390380859375e-05 nb_pixel_total : 44 time to create 1 rle with old method : 0.00011277198791503906 length of segment : 5 time for calcul the mask position with numpy : 4.220008850097656e-05 nb_pixel_total : 720 time to create 1 rle with old method : 0.0008711814880371094 length of segment : 34 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -121.36406 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 10 time for calcul the mask position with numpy : 4.9591064453125e-05 nb_pixel_total : 653 time to create 1 rle with old method : 0.0008027553558349609 length of segment : 46 time for calcul the mask position with numpy : 3.528594970703125e-05 nb_pixel_total : 161 time to create 1 rle with old method : 0.0002753734588623047 length of segment : 17 time for calcul the mask position with numpy : 4.38690185546875e-05 nb_pixel_total : 453 time to create 1 rle with old method : 0.0005948543548583984 length of segment : 34 time for calcul the mask position with numpy : 0.000118255615234375 nb_pixel_total : 1390 time to create 1 rle with old method : 0.0017910003662109375 length of segment : 108 time for calcul the mask position with numpy : 0.0004379749298095703 nb_pixel_total : 14852 time to create 1 rle with old method : 0.01681995391845703 length of segment : 296 time for calcul the mask position with numpy : 7.081031799316406e-05 nb_pixel_total : 1742 time to create 1 rle with old method : 0.002176046371459961 length of segment : 47 time for calcul the mask position with numpy : 5.14984130859375e-05 nb_pixel_total : 1297 time to create 1 rle with old method : 0.0016222000122070312 length of segment : 39 time for calcul the mask position with numpy : 4.2438507080078125e-05 nb_pixel_total : 787 time to create 1 rle with old method : 0.0009999275207519531 length of segment : 34 time for calcul the mask position with numpy : 0.00010752677917480469 nb_pixel_total : 2453 time to create 1 rle with old method : 0.003161907196044922 length of segment : 77 time for calcul the mask position with numpy : 7.033348083496094e-05 nb_pixel_total : 2403 time to create 1 rle with old method : 0.003069639205932617 length of segment : 49 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 141.32656 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 6.771087646484375e-05 nb_pixel_total : 1579 time to create 1 rle with old method : 0.0019478797912597656 length of segment : 84 time for calcul the mask position with numpy : 4.029273986816406e-05 nb_pixel_total : 393 time to create 1 rle with old method : 0.0005214214324951172 length of segment : 32 time for calcul the mask position with numpy : 0.000186920166015625 nb_pixel_total : 3806 time to create 1 rle with old method : 0.006344795227050781 length of segment : 88 time for calcul the mask position with numpy : 8.344650268554688e-05 nb_pixel_total : 1624 time to create 1 rle with old method : 0.002621889114379883 length of segment : 89 time for calcul the mask position with numpy : 3.886222839355469e-05 nb_pixel_total : 170 time to create 1 rle with old method : 0.00033664703369140625 length of segment : 20 time for calcul the mask position with numpy : 0.00021338462829589844 nb_pixel_total : 3067 time to create 1 rle with old method : 0.005212545394897461 length of segment : 191 time for calcul the mask position with numpy : 0.00018644332885742188 nb_pixel_total : 2800 time to create 1 rle with old method : 0.0047872066497802734 length of segment : 181 time for calcul the mask position with numpy : 0.00014066696166992188 nb_pixel_total : 2482 time to create 1 rle with old method : 0.0029573440551757812 length of segment : 159 Processing 1 images image shape: (280, 400, 3) min: 10.00000 max: 187.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 60.84219 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 5.650520324707031e-05 nb_pixel_total : 622 time to create 1 rle with old method : 0.0007798671722412109 length of segment : 62 time for calcul the mask position with numpy : 0.0010192394256591797 nb_pixel_total : 105541 time to create 1 rle with old method : 0.11483263969421387 length of segment : 281 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 148.96719 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 5.9604644775390625e-05 nb_pixel_total : 1610 time to create 1 rle with old method : 0.00188446044921875 length of segment : 90 time for calcul the mask position with numpy : 5.316734313964844e-05 nb_pixel_total : 248 time to create 1 rle with old method : 0.00046324729919433594 length of segment : 22 time for calcul the mask position with numpy : 5.9604644775390625e-05 nb_pixel_total : 1234 time to create 1 rle with old method : 0.0016319751739501953 length of segment : 44 time for calcul the mask position with numpy : 5.078315734863281e-05 nb_pixel_total : 417 time to create 1 rle with old method : 0.0005414485931396484 length of segment : 43 time for calcul the mask position with numpy : 3.838539123535156e-05 nb_pixel_total : 98 time to create 1 rle with old method : 0.00022268295288085938 length of segment : 18 time for calcul the mask position with numpy : 7.748603820800781e-05 nb_pixel_total : 1828 time to create 1 rle with old method : 0.0022711753845214844 length of segment : 76 time for calcul the mask position with numpy : 0.00011682510375976562 nb_pixel_total : 4152 time to create 1 rle with old method : 0.005157947540283203 length of segment : 71 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 36 time for calcul the mask position with numpy : 5.8650970458984375e-05 nb_pixel_total : 155 time to create 1 rle with old method : 0.0002548694610595703 length of segment : 17 time for calcul the mask position with numpy : 4.792213439941406e-05 nb_pixel_total : 504 time to create 1 rle with old method : 0.0006744861602783203 length of segment : 29 time for calcul the mask position with numpy : 6.413459777832031e-05 nb_pixel_total : 1260 time to create 1 rle with old method : 0.0015716552734375 length of segment : 49 time for calcul the mask position with numpy : 3.62396240234375e-05 nb_pixel_total : 133 time to create 1 rle with old method : 0.00018715858459472656 length of segment : 17 time for calcul the mask position with numpy : 3.314018249511719e-05 nb_pixel_total : 276 time to create 1 rle with old method : 0.0004076957702636719 length of segment : 18 time for calcul the mask position with numpy : 3.4332275390625e-05 nb_pixel_total : 180 time to create 1 rle with old method : 0.0002944469451904297 length of segment : 16 time for calcul the mask position with numpy : 7.343292236328125e-05 nb_pixel_total : 834 time to create 1 rle with old method : 0.00159454345703125 length of segment : 39 time for calcul the mask position with numpy : 4.0531158447265625e-05 nb_pixel_total : 200 time to create 1 rle with old method : 0.00031113624572753906 length of segment : 16 time for calcul the mask position with numpy : 3.5762786865234375e-05 nb_pixel_total : 292 time to create 1 rle with old method : 0.00041413307189941406 length of segment : 17 time for calcul the mask position with numpy : 3.7670135498046875e-05 nb_pixel_total : 357 time to create 1 rle with old method : 0.00045871734619140625 length of segment : 31 time for calcul the mask position with numpy : 3.504753112792969e-05 nb_pixel_total : 161 time to create 1 rle with old method : 0.0002396106719970703 length of segment : 21 time for calcul the mask position with numpy : 3.337860107421875e-05 nb_pixel_total : 185 time to create 1 rle with old method : 0.00030922889709472656 length of segment : 16 time for calcul the mask position with numpy : 3.504753112792969e-05 nb_pixel_total : 225 time to create 1 rle with old method : 0.00031065940856933594 length of segment : 25 time for calcul the mask position with numpy : 7.963180541992188e-05 nb_pixel_total : 1627 time to create 1 rle with old method : 0.0030264854431152344 length of segment : 46 time for calcul the mask position with numpy : 4.601478576660156e-05 nb_pixel_total : 377 time to create 1 rle with old method : 0.0005118846893310547 length of segment : 27 time for calcul the mask position with numpy : 4.363059997558594e-05 nb_pixel_total : 662 time to create 1 rle with old method : 0.0008473396301269531 length of segment : 28 time for calcul the mask position with numpy : 4.673004150390625e-05 nb_pixel_total : 831 time to create 1 rle with old method : 0.0010595321655273438 length of segment : 40 time for calcul the mask position with numpy : 3.1948089599609375e-05 nb_pixel_total : 132 time to create 1 rle with old method : 0.00020742416381835938 length of segment : 14 time for calcul the mask position with numpy : 0.0001251697540283203 nb_pixel_total : 2456 time to create 1 rle with old method : 0.002949953079223633 length of segment : 131 time for calcul the mask position with numpy : 3.933906555175781e-05 nb_pixel_total : 58 time to create 1 rle with old method : 0.00011134147644042969 length of segment : 10 time for calcul the mask position with numpy : 3.075599670410156e-05 nb_pixel_total : 126 time to create 1 rle with old method : 0.00019979476928710938 length of segment : 12 time for calcul the mask position with numpy : 3.24249267578125e-05 nb_pixel_total : 176 time to create 1 rle with old method : 0.0002601146697998047 length of segment : 25 time for calcul the mask position with numpy : 3.0040740966796875e-05 nb_pixel_total : 53 time to create 1 rle with old method : 0.00010180473327636719 length of segment : 8 time for calcul the mask position with numpy : 3.314018249511719e-05 nb_pixel_total : 238 time to create 1 rle with old method : 0.00032591819763183594 length of segment : 18 time for calcul the mask position with numpy : 3.528594970703125e-05 nb_pixel_total : 178 time to create 1 rle with old method : 0.00028443336486816406 length of segment : 20 time for calcul the mask position with numpy : 3.0517578125e-05 nb_pixel_total : 159 time to create 1 rle with old method : 0.00024628639221191406 length of segment : 15 time for calcul the mask position with numpy : 3.147125244140625e-05 nb_pixel_total : 195 time to create 1 rle with old method : 0.00029659271240234375 length of segment : 15 time for calcul the mask position with numpy : 3.4809112548828125e-05 nb_pixel_total : 294 time to create 1 rle with old method : 0.0003991127014160156 length of segment : 19 time for calcul the mask position with numpy : 0.00011992454528808594 nb_pixel_total : 2594 time to create 1 rle with old method : 0.003422975540161133 length of segment : 144 time for calcul the mask position with numpy : 8.511543273925781e-05 nb_pixel_total : 1358 time to create 1 rle with old method : 0.0017194747924804688 length of segment : 21 time for calcul the mask position with numpy : 7.486343383789062e-05 nb_pixel_total : 790 time to create 1 rle with old method : 0.001737833023071289 length of segment : 35 time for calcul the mask position with numpy : 5.3882598876953125e-05 nb_pixel_total : 1375 time to create 1 rle with old method : 0.0018608570098876953 length of segment : 21 time for calcul the mask position with numpy : 3.218650817871094e-05 nb_pixel_total : 175 time to create 1 rle with old method : 0.00026917457580566406 length of segment : 14 time for calcul the mask position with numpy : 4.38690185546875e-05 nb_pixel_total : 714 time to create 1 rle with old method : 0.0009124279022216797 length of segment : 30 time for calcul the mask position with numpy : 4.76837158203125e-05 nb_pixel_total : 956 time to create 1 rle with old method : 0.0011272430419921875 length of segment : 41 time for calcul the mask position with numpy : 0.00020503997802734375 nb_pixel_total : 2392 time to create 1 rle with old method : 0.0031943321228027344 length of segment : 129 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 147.43203 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 4.7206878662109375e-05 nb_pixel_total : 882 time to create 1 rle with old method : 0.0011630058288574219 length of segment : 18 time for calcul the mask position with numpy : 4.1961669921875e-05 nb_pixel_total : 745 time to create 1 rle with old method : 0.0009746551513671875 length of segment : 38 time for calcul the mask position with numpy : 8.869171142578125e-05 nb_pixel_total : 3856 time to create 1 rle with old method : 0.004628896713256836 length of segment : 84 time for calcul the mask position with numpy : 9.393692016601562e-05 nb_pixel_total : 457 time to create 1 rle with old method : 0.0007879734039306641 length of segment : 56 time for calcul the mask position with numpy : 4.076957702636719e-05 nb_pixel_total : 331 time to create 1 rle with old method : 0.0005030632019042969 length of segment : 27 time for calcul the mask position with numpy : 9.512901306152344e-05 nb_pixel_total : 4085 time to create 1 rle with old method : 0.00627899169921875 length of segment : 98 time for calcul the mask position with numpy : 7.081031799316406e-05 nb_pixel_total : 261 time to create 1 rle with old method : 0.0003840923309326172 length of segment : 29 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 130.72500 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0004124641418457031 nb_pixel_total : 37935 time to create 1 rle with old method : 0.04094290733337402 length of segment : 233 Detection mask done ! Trying to reset tf kernel 1179625 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 396 tf kernel not reseted sub process len(results) : 230 len(list_Values) 230 None max_time_sub_proc : 3600 parent process len(results) : 0 len(list_Values) 230 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5685 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Catched exception ! Connect or reconnect ! Number saved : None batch 1 Loaded 230 chid ids of type : 4228 Number RLEs to save : 10955 save missing photos in datou_result : time spend for datou_step_exec : 27.41886591911316 time spend to save output : 0.8674590587615967 total time spend for step 1 : 28.286324977874756 step2:brightness Sat Feb 1 18:20:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1738430428_1179035_1333757336_6c8b040489ef1ecc84fb8ddd84cb9c09.jpg treat image : temp/1738430428_1179035_1333757304_96a8babc074521e6cc3f343cd5272cc3.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 2 time used for this insertion : 0.012425899505615234 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 2 time used for this insertion : 0.013459205627441406 save missing photos in datou_result : time spend for datou_step_exec : 0.5292167663574219 time spend to save output : 0.030347108840942383 total time spend for step 2 : 0.5595638751983643 step3:blur_detection Sat Feb 1 18:20:57 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1738430428_1179035_1333757336_6c8b040489ef1ecc84fb8ddd84cb9c09.jpg resize: (1080, 1920) 1333757336 -7.980425247250039 treat image : temp/1738430428_1179035_1333757304_96a8babc074521e6cc3f343cd5272cc3.jpg resize: (1080, 1920) 1333757304 -8.011329491921765 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 2 time used for this insertion : 0.011435270309448242 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 2 time used for this insertion : 0.011452436447143555 save missing photos in datou_result : time spend for datou_step_exec : 2.0227181911468506 time spend to save output : 0.0271608829498291 total time spend for step 3 : 2.0498790740966797 step4:rle_unique_nms_with_priority Sat Feb 1 18:20:59 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 230 chid ids of type : 4228 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 115 nb_hashtags : 7 time to prepare the origin masks : 1.1145803928375244 time for calcul the mask position with numpy : 0.34357762336730957 nb_pixel_total : 1795250 time to create 1 rle with new method : 0.6976087093353271 time for calcul the mask position with numpy : 0.008355855941772461 nb_pixel_total : 1228 time to create 1 rle with old method : 0.0012562274932861328 time for calcul the mask position with numpy : 0.00844573974609375 nb_pixel_total : 1390 time to create 1 rle with old method : 0.0015370845794677734 time for calcul the mask position with numpy : 0.00829625129699707 nb_pixel_total : 1764 time to create 1 rle with old method : 0.0018835067749023438 time for calcul the mask position with numpy : 0.00806283950805664 nb_pixel_total : 44 time to create 1 rle with old method : 8.344650268554688e-05 time for calcul the mask position with numpy : 0.008051156997680664 nb_pixel_total : 74 time to create 1 rle with old method : 9.250640869140625e-05 time for calcul the mask position with numpy : 0.008282661437988281 nb_pixel_total : 111 time to create 1 rle with old method : 0.00012636184692382812 time for calcul the mask position with numpy : 0.008269071578979492 nb_pixel_total : 270 time to create 1 rle with old method : 0.00031566619873046875 time for calcul the mask position with numpy : 0.008411645889282227 nb_pixel_total : 190 time to create 1 rle with old method : 0.00021123886108398438 time for calcul the mask position with numpy : 0.008178472518920898 nb_pixel_total : 52 time to create 1 rle with old method : 9.250640869140625e-05 time for calcul the mask position with numpy : 0.008276700973510742 nb_pixel_total : 948 time to create 1 rle with old method : 0.0010292530059814453 time for calcul the mask position with numpy : 0.008162975311279297 nb_pixel_total : 460 time to create 1 rle with old method : 0.0005106925964355469 time for calcul the mask position with numpy : 0.008336782455444336 nb_pixel_total : 890 time to create 1 rle with old method : 0.0009772777557373047 time for calcul the mask position with numpy : 0.008313894271850586 nb_pixel_total : 30 time to create 1 rle with old method : 4.3392181396484375e-05 time for calcul the mask position with numpy : 0.008169412612915039 nb_pixel_total : 51 time to create 1 rle with old method : 0.00011420249938964844 time for calcul the mask position with numpy : 0.00825810432434082 nb_pixel_total : 56 time to create 1 rle with old method : 6.747245788574219e-05 time for calcul the mask position with numpy : 0.008496522903442383 nb_pixel_total : 34 time to create 1 rle with old method : 4.673004150390625e-05 time for calcul the mask position with numpy : 0.008466958999633789 nb_pixel_total : 281 time to create 1 rle with old method : 0.000347137451171875 time for calcul the mask position with numpy : 0.008522748947143555 nb_pixel_total : 14999 time to create 1 rle with old method : 0.021031856536865234 time for calcul the mask position with numpy : 0.008638143539428711 nb_pixel_total : 922 time to create 1 rle with old method : 0.0010526180267333984 time for calcul the mask position with numpy : 0.008519649505615234 nb_pixel_total : 231 time to create 1 rle with old method : 0.0002758502960205078 time for calcul the mask position with numpy : 0.008611679077148438 nb_pixel_total : 37 time to create 1 rle with old method : 7.772445678710938e-05 time for calcul the mask position with numpy : 0.008510112762451172 nb_pixel_total : 816 time to create 1 rle with old method : 0.0009274482727050781 time for calcul the mask position with numpy : 0.008526086807250977 nb_pixel_total : 81 time to create 1 rle with old method : 0.00011372566223144531 time for calcul the mask position with numpy : 0.008523702621459961 nb_pixel_total : 6101 time to create 1 rle with old method : 0.006680011749267578 time for calcul the mask position with numpy : 0.009009838104248047 nb_pixel_total : 337 time to create 1 rle with old method : 0.0004966259002685547 time for calcul the mask position with numpy : 0.009499311447143555 nb_pixel_total : 8021 time to create 1 rle with old method : 0.008527994155883789 time for calcul the mask position with numpy : 0.008545398712158203 nb_pixel_total : 1587 time to create 1 rle with old method : 0.0021066665649414062 time for calcul the mask position with numpy : 0.00858449935913086 nb_pixel_total : 3243 time to create 1 rle with old method : 0.0035495758056640625 time for calcul the mask position with numpy : 0.008613109588623047 nb_pixel_total : 388 time to create 1 rle with old method : 0.00043487548828125 time for calcul the mask position with numpy : 0.008556604385375977 nb_pixel_total : 58 time to create 1 rle with old method : 0.0001251697540283203 time for calcul the mask position with numpy : 0.008607625961303711 nb_pixel_total : 1581 time to create 1 rle with old method : 0.0017092227935791016 time for calcul the mask position with numpy : 0.008577823638916016 nb_pixel_total : 448 time to create 1 rle with old method : 0.0005304813385009766 time for calcul the mask position with numpy : 0.008640766143798828 nb_pixel_total : 318 time to create 1 rle with old method : 0.0003685951232910156 time for calcul the mask position with numpy : 0.008484363555908203 nb_pixel_total : 15 time to create 1 rle with old method : 3.218650817871094e-05 time for calcul the mask position with numpy : 0.008442878723144531 nb_pixel_total : 1249 time to create 1 rle with old method : 0.0013341903686523438 time for calcul the mask position with numpy : 0.008482694625854492 nb_pixel_total : 289 time to create 1 rle with old method : 0.0003204345703125 time for calcul the mask position with numpy : 0.008464574813842773 nb_pixel_total : 1283 time to create 1 rle with old method : 0.0013556480407714844 time for calcul the mask position with numpy : 0.008450031280517578 nb_pixel_total : 347 time to create 1 rle with old method : 0.0003764629364013672 time for calcul the mask position with numpy : 0.008423328399658203 nb_pixel_total : 1004 time to create 1 rle with old method : 0.001085519790649414 time for calcul the mask position with numpy : 0.0084228515625 nb_pixel_total : 10208 time to create 1 rle with old method : 0.010889530181884766 time for calcul the mask position with numpy : 0.008449316024780273 nb_pixel_total : 732 time to create 1 rle with old method : 0.0008234977722167969 time for calcul the mask position with numpy : 0.008111715316772461 nb_pixel_total : 2571 time to create 1 rle with old method : 0.002901792526245117 time for calcul the mask position with numpy : 0.008404970169067383 nb_pixel_total : 805 time to create 1 rle with old method : 0.0008697509765625 time for calcul the mask position with numpy : 0.008095264434814453 nb_pixel_total : 490 time to create 1 rle with old method : 0.0005929470062255859 time for calcul the mask position with numpy : 0.008279561996459961 nb_pixel_total : 162 time to create 1 rle with old method : 0.0001938343048095703 time for calcul the mask position with numpy : 0.008261442184448242 nb_pixel_total : 170 time to create 1 rle with old method : 0.0001952648162841797 time for calcul the mask position with numpy : 0.008361101150512695 nb_pixel_total : 525 time to create 1 rle with old method : 0.000568389892578125 time for calcul the mask position with numpy : 0.008187055587768555 nb_pixel_total : 1888 time to create 1 rle with old method : 0.002017974853515625 time for calcul the mask position with numpy : 0.008258819580078125 nb_pixel_total : 450 time to create 1 rle with old method : 0.0004985332489013672 time for calcul the mask position with numpy : 0.008232593536376953 nb_pixel_total : 1723 time to create 1 rle with old method : 0.0018858909606933594 time for calcul the mask position with numpy : 0.008115053176879883 nb_pixel_total : 253 time to create 1 rle with old method : 0.0003001689910888672 time for calcul the mask position with numpy : 0.008179426193237305 nb_pixel_total : 663 time to create 1 rle with old method : 0.0007159709930419922 time for calcul the mask position with numpy : 0.008138656616210938 nb_pixel_total : 20 time to create 1 rle with old method : 4.029273986816406e-05 time for calcul the mask position with numpy : 0.008180856704711914 nb_pixel_total : 137 time to create 1 rle with old method : 0.00014472007751464844 time for calcul the mask position with numpy : 0.008095502853393555 nb_pixel_total : 301 time to create 1 rle with old method : 0.00034689903259277344 time for calcul the mask position with numpy : 0.009879589080810547 nb_pixel_total : 1951 time to create 1 rle with old method : 0.0031931400299072266 time for calcul the mask position with numpy : 0.00875091552734375 nb_pixel_total : 153 time to create 1 rle with old method : 0.00019431114196777344 time for calcul the mask position with numpy : 0.008374691009521484 nb_pixel_total : 725 time to create 1 rle with old method : 0.0008420944213867188 time for calcul the mask position with numpy : 0.008109331130981445 nb_pixel_total : 153 time to create 1 rle with old method : 0.0001647472381591797 time for calcul the mask position with numpy : 0.008861303329467773 nb_pixel_total : 106548 time to create 1 rle with old method : 0.10936570167541504 time for calcul the mask position with numpy : 0.008193731307983398 nb_pixel_total : 846 time to create 1 rle with old method : 0.0009717941284179688 time for calcul the mask position with numpy : 0.008182287216186523 nb_pixel_total : 79 time to create 1 rle with old method : 0.00010037422180175781 time for calcul the mask position with numpy : 0.008020877838134766 nb_pixel_total : 711 time to create 1 rle with old method : 0.0008063316345214844 time for calcul the mask position with numpy : 0.008334159851074219 nb_pixel_total : 10 time to create 1 rle with old method : 2.9087066650390625e-05 time for calcul the mask position with numpy : 0.008249282836914062 nb_pixel_total : 193 time to create 1 rle with old method : 0.00023055076599121094 time for calcul the mask position with numpy : 0.008306026458740234 nb_pixel_total : 30630 time to create 1 rle with old method : 0.0319371223449707 time for calcul the mask position with numpy : 0.008492231369018555 nb_pixel_total : 144 time to create 1 rle with old method : 0.0001735687255859375 time for calcul the mask position with numpy : 0.008208513259887695 nb_pixel_total : 260 time to create 1 rle with old method : 0.0002689361572265625 time for calcul the mask position with numpy : 0.008377790451049805 nb_pixel_total : 990 time to create 1 rle with old method : 0.0011017322540283203 time for calcul the mask position with numpy : 0.008346319198608398 nb_pixel_total : 231 time to create 1 rle with old method : 0.00025534629821777344 time for calcul the mask position with numpy : 0.00821685791015625 nb_pixel_total : 781 time to create 1 rle with old method : 0.0008459091186523438 time for calcul the mask position with numpy : 0.00820159912109375 nb_pixel_total : 1277 time to create 1 rle with old method : 0.001463174819946289 time for calcul the mask position with numpy : 0.008457183837890625 nb_pixel_total : 1667 time to create 1 rle with old method : 0.0017843246459960938 time for calcul the mask position with numpy : 0.008521080017089844 nb_pixel_total : 40950 time to create 1 rle with old method : 0.04319477081298828 time for calcul the mask position with numpy : 0.008443832397460938 nb_pixel_total : 383 time to create 1 rle with old method : 0.0004558563232421875 time for calcul the mask position with numpy : 0.008474588394165039 nb_pixel_total : 237 time to create 1 rle with old method : 0.0002703666687011719 time for calcul the mask position with numpy : 0.008440494537353516 nb_pixel_total : 153 time to create 1 rle with old method : 0.0001761913299560547 time for calcul the mask position with numpy : 0.008452892303466797 nb_pixel_total : 147 time to create 1 rle with old method : 0.0001704692840576172 time for calcul the mask position with numpy : 0.008409500122070312 nb_pixel_total : 1547 time to create 1 rle with old method : 0.0016679763793945312 time for calcul the mask position with numpy : 0.00833439826965332 nb_pixel_total : 235 time to create 1 rle with old method : 0.0002593994140625 time for calcul the mask position with numpy : 0.008121252059936523 nb_pixel_total : 317 time to create 1 rle with old method : 0.0003414154052734375 time for calcul the mask position with numpy : 0.008292198181152344 nb_pixel_total : 214 time to create 1 rle with old method : 0.00025177001953125 time for calcul the mask position with numpy : 0.008247137069702148 nb_pixel_total : 458 time to create 1 rle with old method : 0.0005385875701904297 time for calcul the mask position with numpy : 0.008304595947265625 nb_pixel_total : 287 time to create 1 rle with old method : 0.0003161430358886719 time for calcul the mask position with numpy : 0.00829625129699707 nb_pixel_total : 51 time to create 1 rle with old method : 0.00010156631469726562 time for calcul the mask position with numpy : 0.008343696594238281 nb_pixel_total : 2267 time to create 1 rle with old method : 0.0025365352630615234 time for calcul the mask position with numpy : 0.008234500885009766 nb_pixel_total : 614 time to create 1 rle with old method : 0.0006129741668701172 time for calcul the mask position with numpy : 0.00806427001953125 nb_pixel_total : 250 time to create 1 rle with old method : 0.0002589225769042969 time for calcul the mask position with numpy : 0.008172273635864258 nb_pixel_total : 31 time to create 1 rle with old method : 5.030632019042969e-05 time for calcul the mask position with numpy : 0.008158206939697266 nb_pixel_total : 11 time to create 1 rle with old method : 2.5033950805664062e-05 time for calcul the mask position with numpy : 0.008235454559326172 nb_pixel_total : 244 time to create 1 rle with old method : 0.0002498626708984375 time for calcul the mask position with numpy : 0.008257150650024414 nb_pixel_total : 428 time to create 1 rle with old method : 0.0004968643188476562 time for calcul the mask position with numpy : 0.00845789909362793 nb_pixel_total : 110 time to create 1 rle with old method : 0.00013256072998046875 time for calcul the mask position with numpy : 0.008464813232421875 nb_pixel_total : 39 time to create 1 rle with old method : 6.961822509765625e-05 time for calcul the mask position with numpy : 0.008359432220458984 nb_pixel_total : 1580 time to create 1 rle with old method : 0.0017893314361572266 time for calcul the mask position with numpy : 0.008168697357177734 nb_pixel_total : 197 time to create 1 rle with old method : 0.0002205371856689453 time for calcul the mask position with numpy : 0.008095979690551758 nb_pixel_total : 105 time to create 1 rle with old method : 0.00011754035949707031 time for calcul the mask position with numpy : 0.008047342300415039 nb_pixel_total : 400 time to create 1 rle with old method : 0.00044846534729003906 time for calcul the mask position with numpy : 0.008165359497070312 nb_pixel_total : 296 time to create 1 rle with old method : 0.0003006458282470703 time for calcul the mask position with numpy : 0.008090972900390625 nb_pixel_total : 387 time to create 1 rle with old method : 0.000396728515625 time for calcul the mask position with numpy : 0.008054733276367188 nb_pixel_total : 97 time to create 1 rle with old method : 0.00010633468627929688 time for calcul the mask position with numpy : 0.008179664611816406 nb_pixel_total : 696 time to create 1 rle with old method : 0.0007195472717285156 time for calcul the mask position with numpy : 0.008521080017089844 nb_pixel_total : 668 time to create 1 rle with old method : 0.0007915496826171875 time for calcul the mask position with numpy : 0.008495330810546875 nb_pixel_total : 179 time to create 1 rle with old method : 0.0002124309539794922 time for calcul the mask position with numpy : 0.008496999740600586 nb_pixel_total : 834 time to create 1 rle with old method : 0.00091552734375 time for calcul the mask position with numpy : 0.008545160293579102 nb_pixel_total : 798 time to create 1 rle with old method : 0.0008890628814697266 time for calcul the mask position with numpy : 0.00851750373840332 nb_pixel_total : 1279 time to create 1 rle with old method : 0.0013883113861083984 time for calcul the mask position with numpy : 0.008569002151489258 nb_pixel_total : 73 time to create 1 rle with old method : 9.942054748535156e-05 time for calcul the mask position with numpy : 0.008542537689208984 nb_pixel_total : 74 time to create 1 rle with old method : 9.465217590332031e-05 time for calcul the mask position with numpy : 0.008559942245483398 nb_pixel_total : 7 time to create 1 rle with old method : 2.86102294921875e-05 time for calcul the mask position with numpy : 0.008532285690307617 nb_pixel_total : 489 time to create 1 rle with old method : 0.0005583763122558594 time for calcul the mask position with numpy : 0.008514165878295898 nb_pixel_total : 59 time to create 1 rle with old method : 9.059906005859375e-05 time for calcul the mask position with numpy : 0.00816035270690918 nb_pixel_total : 1364 time to create 1 rle with old method : 0.001355886459350586 time for calcul the mask position with numpy : 0.008081912994384766 nb_pixel_total : 79 time to create 1 rle with old method : 8.869171142578125e-05 time for calcul the mask position with numpy : 0.008121013641357422 nb_pixel_total : 41 time to create 1 rle with old method : 0.00011920928955078125 create new chi : 2.3169615268707275 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.005401134490966797 batch 1 Loaded 116 chid ids of type : 4230 Number RLEs to save : 10690 TO DO : save crop sub photo not yet done ! save time : 0.6235153675079346 nb_obj : 115 nb_hashtags : 8 time to prepare the origin masks : 2.125636100769043 time for calcul the mask position with numpy : 0.7136175632476807 nb_pixel_total : 1820614 time to create 1 rle with new method : 1.2032036781311035 time for calcul the mask position with numpy : 0.008766651153564453 nb_pixel_total : 955 time to create 1 rle with old method : 0.001058816909790039 time for calcul the mask position with numpy : 0.006934404373168945 nb_pixel_total : 667 time to create 1 rle with old method : 0.0006744861602783203 time for calcul the mask position with numpy : 0.00559234619140625 nb_pixel_total : 914 time to create 1 rle with old method : 0.00095367431640625 time for calcul the mask position with numpy : 0.005880117416381836 nb_pixel_total : 62 time to create 1 rle with old method : 8.821487426757812e-05 time for calcul the mask position with numpy : 0.0058329105377197266 nb_pixel_total : 95 time to create 1 rle with old method : 0.00011587142944335938 time for calcul the mask position with numpy : 0.0059452056884765625 nb_pixel_total : 1048 time to create 1 rle with old method : 0.0010793209075927734 time for calcul the mask position with numpy : 0.005576133728027344 nb_pixel_total : 412 time to create 1 rle with old method : 0.00041985511779785156 time for calcul the mask position with numpy : 0.0055904388427734375 nb_pixel_total : 508 time to create 1 rle with old method : 0.0005092620849609375 time for calcul the mask position with numpy : 0.005800008773803711 nb_pixel_total : 27 time to create 1 rle with old method : 6.389617919921875e-05 time for calcul the mask position with numpy : 0.005883455276489258 nb_pixel_total : 1321 time to create 1 rle with old method : 0.0012950897216796875 time for calcul the mask position with numpy : 0.005570411682128906 nb_pixel_total : 909 time to create 1 rle with old method : 0.0008995532989501953 time for calcul the mask position with numpy : 0.005591630935668945 nb_pixel_total : 236 time to create 1 rle with old method : 0.00024509429931640625 time for calcul the mask position with numpy : 0.005461215972900391 nb_pixel_total : 1007 time to create 1 rle with old method : 0.0010170936584472656 time for calcul the mask position with numpy : 0.0056324005126953125 nb_pixel_total : 1052 time to create 1 rle with old method : 0.001028299331665039 time for calcul the mask position with numpy : 0.005536794662475586 nb_pixel_total : 147 time to create 1 rle with old method : 0.0001761913299560547 time for calcul the mask position with numpy : 0.0055272579193115234 nb_pixel_total : 59 time to create 1 rle with old method : 7.796287536621094e-05 time for calcul the mask position with numpy : 0.005657196044921875 nb_pixel_total : 6799 time to create 1 rle with old method : 0.006913423538208008 time for calcul the mask position with numpy : 0.006006002426147461 nb_pixel_total : 349 time to create 1 rle with old method : 0.0004100799560546875 time for calcul the mask position with numpy : 0.0059430599212646484 nb_pixel_total : 8180 time to create 1 rle with old method : 0.008420228958129883 time for calcul the mask position with numpy : 0.00603485107421875 nb_pixel_total : 91 time to create 1 rle with old method : 0.0001251697540283203 time for calcul the mask position with numpy : 0.005802154541015625 nb_pixel_total : 150 time to create 1 rle with old method : 0.0001709461212158203 time for calcul the mask position with numpy : 0.005965709686279297 nb_pixel_total : 659 time to create 1 rle with old method : 0.0007383823394775391 time for calcul the mask position with numpy : 0.005948781967163086 nb_pixel_total : 74 time to create 1 rle with old method : 0.00011467933654785156 time for calcul the mask position with numpy : 0.005925178527832031 nb_pixel_total : 472 time to create 1 rle with old method : 0.0004820823669433594 time for calcul the mask position with numpy : 0.005765438079833984 nb_pixel_total : 358 time to create 1 rle with old method : 0.00042748451232910156 time for calcul the mask position with numpy : 0.005914926528930664 nb_pixel_total : 11 time to create 1 rle with old method : 5.316734313964844e-05 time for calcul the mask position with numpy : 0.005953073501586914 nb_pixel_total : 1233 time to create 1 rle with old method : 0.0012676715850830078 time for calcul the mask position with numpy : 0.005885601043701172 nb_pixel_total : 14710 time to create 1 rle with old method : 0.014641046524047852 time for calcul the mask position with numpy : 0.006173133850097656 nb_pixel_total : 720 time to create 1 rle with old method : 0.0007541179656982422 time for calcul the mask position with numpy : 0.006020069122314453 nb_pixel_total : 787 time to create 1 rle with old method : 0.0008504390716552734 time for calcul the mask position with numpy : 0.00580906867980957 nb_pixel_total : 1273 time to create 1 rle with old method : 0.001283407211303711 time for calcul the mask position with numpy : 0.006170749664306641 nb_pixel_total : 170 time to create 1 rle with old method : 0.0001983642578125 time for calcul the mask position with numpy : 0.005928516387939453 nb_pixel_total : 393 time to create 1 rle with old method : 0.00042319297790527344 time for calcul the mask position with numpy : 0.005815744400024414 nb_pixel_total : 834 time to create 1 rle with old method : 0.0009295940399169922 time for calcul the mask position with numpy : 0.0071544647216796875 nb_pixel_total : 10503 time to create 1 rle with old method : 0.01192927360534668 time for calcul the mask position with numpy : 0.005995750427246094 nb_pixel_total : 513 time to create 1 rle with old method : 0.000568389892578125 time for calcul the mask position with numpy : 0.006022214889526367 nb_pixel_total : 3067 time to create 1 rle with old method : 0.0030367374420166016 time for calcul the mask position with numpy : 0.0056416988372802734 nb_pixel_total : 1390 time to create 1 rle with old method : 0.0014503002166748047 time for calcul the mask position with numpy : 0.005708932876586914 nb_pixel_total : 89 time to create 1 rle with old method : 0.00013065338134765625 time for calcul the mask position with numpy : 0.006135225296020508 nb_pixel_total : 571 time to create 1 rle with old method : 0.0006382465362548828 time for calcul the mask position with numpy : 0.005856513977050781 nb_pixel_total : 187 time to create 1 rle with old method : 0.0002574920654296875 time for calcul the mask position with numpy : 0.006214618682861328 nb_pixel_total : 161 time to create 1 rle with old method : 0.00019669532775878906 time for calcul the mask position with numpy : 0.006562471389770508 nb_pixel_total : 97 time to create 1 rle with old method : 0.00013756752014160156 time for calcul the mask position with numpy : 0.0062694549560546875 nb_pixel_total : 810 time to create 1 rle with old method : 0.0008244514465332031 time for calcul the mask position with numpy : 0.005828857421875 nb_pixel_total : 3806 time to create 1 rle with old method : 0.00390934944152832 time for calcul the mask position with numpy : 0.005980730056762695 nb_pixel_total : 443 time to create 1 rle with old method : 0.0004742145538330078 time for calcul the mask position with numpy : 0.005733013153076172 nb_pixel_total : 459 time to create 1 rle with old method : 0.0005147457122802734 time for calcul the mask position with numpy : 0.005890846252441406 nb_pixel_total : 453 time to create 1 rle with old method : 0.0005092620849609375 time for calcul the mask position with numpy : 0.0055999755859375 nb_pixel_total : 2453 time to create 1 rle with old method : 0.0025053024291992188 time for calcul the mask position with numpy : 0.005711078643798828 nb_pixel_total : 143 time to create 1 rle with old method : 0.00023245811462402344 time for calcul the mask position with numpy : 0.0055730342864990234 nb_pixel_total : 376 time to create 1 rle with old method : 0.0004220008850097656 time for calcul the mask position with numpy : 0.0056917667388916016 nb_pixel_total : 1579 time to create 1 rle with old method : 0.0015902519226074219 time for calcul the mask position with numpy : 0.0057811737060546875 nb_pixel_total : 653 time to create 1 rle with old method : 0.0006535053253173828 time for calcul the mask position with numpy : 0.0057849884033203125 nb_pixel_total : 112 time to create 1 rle with old method : 0.0001327991485595703 time for calcul the mask position with numpy : 0.005612373352050781 nb_pixel_total : 24 time to create 1 rle with old method : 5.14984130859375e-05 time for calcul the mask position with numpy : 0.005837440490722656 nb_pixel_total : 21 time to create 1 rle with old method : 5.3882598876953125e-05 time for calcul the mask position with numpy : 0.006046772003173828 nb_pixel_total : 82 time to create 1 rle with old method : 0.00010061264038085938 time for calcul the mask position with numpy : 0.00614166259765625 nb_pixel_total : 126 time to create 1 rle with old method : 0.0001621246337890625 time for calcul the mask position with numpy : 0.006589412689208984 nb_pixel_total : 313 time to create 1 rle with old method : 0.0003342628479003906 time for calcul the mask position with numpy : 0.005909919738769531 nb_pixel_total : 1742 time to create 1 rle with old method : 0.0017898082733154297 time for calcul the mask position with numpy : 0.005877971649169922 nb_pixel_total : 794 time to create 1 rle with old method : 0.0008459091186523438 time for calcul the mask position with numpy : 0.005732297897338867 nb_pixel_total : 141 time to create 1 rle with old method : 0.00015783309936523438 time for calcul the mask position with numpy : 0.006823062896728516 nb_pixel_total : 105157 time to create 1 rle with old method : 0.10305404663085938 time for calcul the mask position with numpy : 0.007002353668212891 nb_pixel_total : 882 time to create 1 rle with old method : 0.000978708267211914 time for calcul the mask position with numpy : 0.006636857986450195 nb_pixel_total : 331 time to create 1 rle with old method : 0.00040030479431152344 time for calcul the mask position with numpy : 0.0071332454681396484 nb_pixel_total : 195 time to create 1 rle with old method : 0.00021338462829589844 time for calcul the mask position with numpy : 0.006613254547119141 nb_pixel_total : 3856 time to create 1 rle with old method : 0.004052162170410156 time for calcul the mask position with numpy : 0.007056713104248047 nb_pixel_total : 357 time to create 1 rle with old method : 0.0004019737243652344 time for calcul the mask position with numpy : 0.006773710250854492 nb_pixel_total : 133 time to create 1 rle with old method : 0.00016307830810546875 time for calcul the mask position with numpy : 0.0065555572509765625 nb_pixel_total : 178 time to create 1 rle with old method : 0.00021910667419433594 time for calcul the mask position with numpy : 0.007288455963134766 nb_pixel_total : 37935 time to create 1 rle with old method : 0.03817152976989746 time for calcul the mask position with numpy : 0.0074656009674072266 nb_pixel_total : 58 time to create 1 rle with old method : 8.082389831542969e-05 time for calcul the mask position with numpy : 0.007566213607788086 nb_pixel_total : 1626 time to create 1 rle with old method : 0.0015993118286132812 time for calcul the mask position with numpy : 0.006999969482421875 nb_pixel_total : 417 time to create 1 rle with old method : 0.00042247772216796875 time for calcul the mask position with numpy : 0.009762287139892578 nb_pixel_total : 155 time to create 1 rle with old method : 0.00020956993103027344 time for calcul the mask position with numpy : 0.009322881698608398 nb_pixel_total : 831 time to create 1 rle with old method : 0.0008952617645263672 time for calcul the mask position with numpy : 0.007703304290771484 nb_pixel_total : 126 time to create 1 rle with old method : 0.00032711029052734375 time for calcul the mask position with numpy : 0.007798671722412109 nb_pixel_total : 132 time to create 1 rle with old method : 0.00016450881958007812 time for calcul the mask position with numpy : 0.007000446319580078 nb_pixel_total : 161 time to create 1 rle with old method : 0.00020313262939453125 time for calcul the mask position with numpy : 0.00923466682434082 nb_pixel_total : 225 time to create 1 rle with old method : 0.0002377033233642578 time for calcul the mask position with numpy : 0.008485078811645508 nb_pixel_total : 175 time to create 1 rle with old method : 0.00019550323486328125 time for calcul the mask position with numpy : 0.008528947830200195 nb_pixel_total : 185 time to create 1 rle with old method : 0.000209808349609375 time for calcul the mask position with numpy : 0.008061408996582031 nb_pixel_total : 248 time to create 1 rle with old method : 0.00028634071350097656 time for calcul the mask position with numpy : 0.007082700729370117 nb_pixel_total : 332 time to create 1 rle with old method : 0.00040340423583984375 time for calcul the mask position with numpy : 0.0072138309478759766 nb_pixel_total : 1234 time to create 1 rle with old method : 0.0013339519500732422 time for calcul the mask position with numpy : 0.006505250930786133 nb_pixel_total : 2454 time to create 1 rle with old method : 0.0025942325592041016 time for calcul the mask position with numpy : 0.007176876068115234 nb_pixel_total : 53 time to create 1 rle with old method : 7.605552673339844e-05 time for calcul the mask position with numpy : 0.006052494049072266 nb_pixel_total : 745 time to create 1 rle with old method : 0.0008356571197509766 time for calcul the mask position with numpy : 0.006100893020629883 nb_pixel_total : 33 time to create 1 rle with old method : 8.845329284667969e-05 time for calcul the mask position with numpy : 0.006196737289428711 nb_pixel_total : 276 time to create 1 rle with old method : 0.0003273487091064453 time for calcul the mask position with numpy : 0.006025552749633789 nb_pixel_total : 238 time to create 1 rle with old method : 0.0002543926239013672 time for calcul the mask position with numpy : 0.0058100223541259766 nb_pixel_total : 176 time to create 1 rle with old method : 0.00019979476928710938 time for calcul the mask position with numpy : 0.005864381790161133 nb_pixel_total : 1610 time to create 1 rle with old method : 0.0016684532165527344 time for calcul the mask position with numpy : 0.006103515625 nb_pixel_total : 200 time to create 1 rle with old method : 0.00021839141845703125 time for calcul the mask position with numpy : 0.0057032108306884766 nb_pixel_total : 159 time to create 1 rle with old method : 0.0002033710479736328 time for calcul the mask position with numpy : 0.005834341049194336 nb_pixel_total : 331 time to create 1 rle with old method : 0.0003390312194824219 time for calcul the mask position with numpy : 0.005758047103881836 nb_pixel_total : 292 time to create 1 rle with old method : 0.00030422210693359375 time for calcul the mask position with numpy : 0.005781412124633789 nb_pixel_total : 377 time to create 1 rle with old method : 0.00038933753967285156 time for calcul the mask position with numpy : 0.006348371505737305 nb_pixel_total : 126 time to create 1 rle with old method : 0.0001575946807861328 time for calcul the mask position with numpy : 0.006447792053222656 nb_pixel_total : 4152 time to create 1 rle with old method : 0.004460811614990234 time for calcul the mask position with numpy : 0.005859851837158203 nb_pixel_total : 294 time to create 1 rle with old method : 0.0003085136413574219 time for calcul the mask position with numpy : 0.006128072738647461 nb_pixel_total : 58 time to create 1 rle with old method : 9.703636169433594e-05 time for calcul the mask position with numpy : 0.005625486373901367 nb_pixel_total : 662 time to create 1 rle with old method : 0.0006744861602783203 time for calcul the mask position with numpy : 0.005502223968505859 nb_pixel_total : 1828 time to create 1 rle with old method : 0.0017578601837158203 time for calcul the mask position with numpy : 0.005717754364013672 nb_pixel_total : 180 time to create 1 rle with old method : 0.00019788742065429688 time for calcul the mask position with numpy : 0.005694866180419922 nb_pixel_total : 457 time to create 1 rle with old method : 0.0005028247833251953 time for calcul the mask position with numpy : 0.006148576736450195 nb_pixel_total : 622 time to create 1 rle with old method : 0.0006213188171386719 time for calcul the mask position with numpy : 0.005924701690673828 nb_pixel_total : 834 time to create 1 rle with old method : 0.0009112358093261719 time for calcul the mask position with numpy : 0.005843639373779297 nb_pixel_total : 790 time to create 1 rle with old method : 0.00078582763671875 time for calcul the mask position with numpy : 0.0065686702728271484 nb_pixel_total : 1260 time to create 1 rle with old method : 0.0013432502746582031 time for calcul the mask position with numpy : 0.006289958953857422 nb_pixel_total : 98 time to create 1 rle with old method : 0.00011587142944335938 time for calcul the mask position with numpy : 0.006224155426025391 nb_pixel_total : 57 time to create 1 rle with old method : 7.867813110351562e-05 time for calcul the mask position with numpy : 0.005856037139892578 nb_pixel_total : 504 time to create 1 rle with old method : 0.0005195140838623047 time for calcul the mask position with numpy : 0.006025552749633789 nb_pixel_total : 1358 time to create 1 rle with old method : 0.0013394355773925781 time for calcul the mask position with numpy : 0.008249521255493164 nb_pixel_total : 43 time to create 1 rle with old method : 0.00010561943054199219 create new chi : 2.912252187728882 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.0012373924255371094 batch 1 Loaded 116 chid ids of type : 4230 Number RLEs to save : 10918 TO DO : save crop sub photo not yet done ! save time : 0.6254613399505615 map_output_result : {1333757336: (0.0, 'Should be the crop_list due to order', 0.0), 1333757304: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1333757336, 1333757304] Looping around the photos to save general results len do output : 2 /1333757336.Didn't retrieve data . /1333757304.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2537864') ('4234', '20143824', '1333757336', None, None, None, None, None, '2537864') ('4234', None, None, None, None, None, None, None, '2537864') ('4234', '20143824', '1333757304', None, None, None, None, None, '2537864') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.01198887825012207 save_final save missing photos in datou_result : time spend for datou_step_exec : 10.113094568252563 time spend to save output : 0.012195110321044922 total time spend for step 4 : 10.125289678573608 step5:crop_condition Sat Feb 1 18:21:09 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4230 Loading chi in step crop for list_pids : 2 ! batch 1 Loaded 232 chid ids of type : 4230 begin to crop the class : papier param for this class : {'min_score': 0.6} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 90 About to insert : list_path_to_insert length 90 new photo from crops ! About to upload 90 photos upload in portfolio : 4869462 init cache_photo without model_param we have 90 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738430471_1179035 we have uploaded 90 photos in the portfolio 4869462 time of upload the photos Elapsed time : 26.380399227142334 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.6} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 30 About to insert : list_path_to_insert length 30 new photo from crops ! About to upload 30 photos upload in portfolio : 4869462 init cache_photo without model_param we have 30 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738430498_1179035 we have uploaded 30 photos in the portfolio 4869462 time of upload the photos Elapsed time : 7.602131128311157 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.6} filtre for class : metal hashtag_id of this class : 492628673 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738430505_1179035 we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.7624890804290771 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.6} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738430506_1179035 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.121654748916626 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.6} filtre for class : autre hashtag_id of this class : 494826614 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 31 About to insert : list_path_to_insert length 31 new photo from crops ! About to upload 31 photos upload in portfolio : 4869462 init cache_photo without model_param we have 31 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738430509_1179035 we have uploaded 31 photos in the portfolio 4869462 time of upload the photos Elapsed time : 7.067697525024414 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.6} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.6} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1333757336, 1333757304] Looping around the photos to save general results len do output : 156 /1333762521Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762524Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762525Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762526Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762527Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762528Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762529Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762530Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762531Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762532Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762533Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762534Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762535Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762536Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762537Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762538Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762539Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762540Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762541Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762542Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762543Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762544Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762545Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762546Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762547Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762548Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762549Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762550Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762551Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762552Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762553Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762554Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762555Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762556Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762557Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762558Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762559Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762560Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762561Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762562Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762563Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762564Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762565Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762566Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762567Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762569Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762570Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762571Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762572Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762573Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762574Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762575Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762576Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762578Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762579Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762580Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762581Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762582Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762583Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762584Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762585Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762586Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762587Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762588Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762589Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762590Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762591Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762592Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762593Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762594Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762595Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762596Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762597Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762598Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762599Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762600Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762601Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762602Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762603Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762604Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762605Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762606Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762607Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762608Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762609Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762610Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762611Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762612Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762613Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762614Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762621Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762622Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762623Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762624Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762625Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762626Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762627Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762628Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762629Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762630Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762631Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762632Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762633Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762634Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762635Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762636Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762637Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762638Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762639Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762640Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762641Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762642Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762643Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762644Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762645Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762646Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762647Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762648Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762649Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762650Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762653Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762654Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762656Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762657Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762658Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762664Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762665Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762666Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762667Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762668Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762669Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762670Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762671Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762672Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762673Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762674Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762675Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762676Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762677Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762678Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762679Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762681Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762682Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762683Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762684Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762685Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762686Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762687Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762688Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762689Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762690Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762691Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762692Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762693Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762694Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1333762695Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2537864') ('4234', '20143824', '1333757336', None, None, None, None, None, '2537864') ('4234', None, None, None, None, None, None, None, '2537864') ('4234', '20143824', '1333757304', None, None, None, None, None, '2537864') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 470 time used for this insertion : 0.030841827392578125 save_final save missing photos in datou_result : time spend for datou_step_exec : 46.72416543960571 time spend to save output : 0.03622770309448242 total time spend for step 5 : 46.760393142700195 step6:thcl Sat Feb 1 18:21:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! we are using the classfication for only one thcl 3237 time to import caffe and check if the image exist : 0.006971836090087891 time to convert the images to numpy array : 0.03652834892272949 time to import caffe and check if the image exist : 0.011598348617553711 time to convert the images to numpy array : 0.0356602668762207 time to import caffe and check if the image exist : 0.005632638931274414 time to convert the images to numpy array : 0.04317450523376465 time to import caffe and check if the image exist : 0.013156414031982422 time to convert the images to numpy array : 0.03908133506774902 time to import caffe and check if the image exist : 0.01099705696105957 time to convert the images to numpy array : 0.04653310775756836 time to import caffe and check if the image exist : 0.012188434600830078 time to convert the images to numpy array : 0.045363664627075195 time to import caffe and check if the image exist : 0.020238876342773438 time to convert the images to numpy array : 0.03898358345031738 time to import caffe and check if the image exist : 0.010467052459716797 time to convert the images to numpy array : 0.04942178726196289 time to import caffe and check if the image exist : 0.01185750961303711 time to convert the images to numpy array : 0.0494084358215332 time to import caffe and check if the image exist : 0.011708974838256836 time to convert the images to numpy array : 0.04929995536804199 total time to convert the images to numpy array : 0.32816505432128906 list photo_ids error: [] list photo_ids correct : [1333762589, 1333762590, 1333762591, 1333762592, 1333762593, 1333762594, 1333762595, 1333762596, 1333762597, 1333762598, 1333762599, 1333762600, 1333762601, 1333762602, 1333762603, 1333762604, 1333762572, 1333762573, 1333762574, 1333762575, 1333762576, 1333762578, 1333762579, 1333762580, 1333762581, 1333762582, 1333762583, 1333762584, 1333762585, 1333762586, 1333762587, 1333762588, 1333762684, 1333762685, 1333762686, 1333762687, 1333762688, 1333762689, 1333762690, 1333762691, 1333762692, 1333762693, 1333762694, 1333762695, 1333762539, 1333762540, 1333762541, 1333762542, 1333762543, 1333762544, 1333762545, 1333762546, 1333762547, 1333762548, 1333762549, 1333762550, 1333762551, 1333762552, 1333762553, 1333762554, 1333762521, 1333762524, 1333762525, 1333762526, 1333762527, 1333762528, 1333762529, 1333762530, 1333762531, 1333762532, 1333762533, 1333762534, 1333762535, 1333762536, 1333762537, 1333762538, 1333762667, 1333762668, 1333762669, 1333762670, 1333762671, 1333762672, 1333762673, 1333762674, 1333762675, 1333762676, 1333762677, 1333762678, 1333762679, 1333762681, 1333762682, 1333762683, 1333762627, 1333762628, 1333762629, 1333762630, 1333762631, 1333762632, 1333762633, 1333762634, 1333762635, 1333762636, 1333762637, 1333762638, 1333762639, 1333762640, 1333762641, 1333762642, 1333762605, 1333762606, 1333762607, 1333762608, 1333762609, 1333762610, 1333762611, 1333762612, 1333762613, 1333762614, 1333762621, 1333762622, 1333762623, 1333762624, 1333762625, 1333762626, 1333762555, 1333762556, 1333762557, 1333762558, 1333762559, 1333762560, 1333762561, 1333762562, 1333762563, 1333762564, 1333762565, 1333762566, 1333762567, 1333762569, 1333762570, 1333762571, 1333762643, 1333762644, 1333762645, 1333762646, 1333762647, 1333762648, 1333762649, 1333762650, 1333762653, 1333762654, 1333762656, 1333762657, 1333762658, 1333762664, 1333762665, 1333762666] number of photos to traite : 156 try to delete the photos incorrect in DB tagging for thcl : 3237 To do loadFromThcl(), then load ParamDescType : thcl3237 thcls : [{'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'}] thcl {'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'} Update svm_hashtag_type_desc : 5561 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) To loadFromThcl() : net_5561 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 5464 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_rubbia_refus_2500 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_rubbia_refus_2500 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_rubbia_refus_2500 /data/models_weight/learn_rubbia_refus_2500/caffemodel size_local : 94358479 size in s3 : 94358479 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:16 caffemodel already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt size_local : 32544 size in s3 : 32544 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:15 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:09:52 mean.npy already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/synset_words.txt size_local : 334 size in s3 : 334 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:10:06 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt caffemodel_filename : /data/models_weight/learn_rubbia_refus_2500/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 5245 max_wait_temp : 1 max_wait : 0 dict_keys(['pool5', 'prob']) time used to do the prepocess of the images : 0.7927522659301758 time used to do the prediction : 0.8029868602752686 save descriptor for thcl : 3237 time to traite the descriptors : 0.9977381229400635 Catched exception ! Connect or reconnect ! storage_type for insertDescriptorsMulti : 3 To insert : 1333762589 To insert : 1333762590 To insert : 1333762591 To insert : 1333762592 To insert : 1333762593 To insert : 1333762594 To insert : 1333762595 To insert : 1333762596 To insert : 1333762597 To insert : 1333762598 To insert : 1333762599 To insert : 1333762600 To insert : 1333762601 To insert : 1333762602 To insert : 1333762603 To insert : 1333762604 To insert : 1333762572 To insert : 1333762573 To insert : 1333762574 To insert : 1333762575 To insert : 1333762576 To insert : 1333762578 To insert : 1333762579 To insert : 1333762580 To insert : 1333762581 To insert : 1333762582 To insert : 1333762583 To insert : 1333762584 To insert : 1333762585 To insert : 1333762586 To insert : 1333762587 To insert : 1333762588 To insert : 1333762684 To insert : 1333762685 To insert : 1333762686 To insert : 1333762687 To insert : 1333762688 To insert : 1333762689 To insert : 1333762690 To insert : 1333762691 To insert : 1333762692 To insert : 1333762693 To insert : 1333762694 To insert : 1333762695 To insert : 1333762539 To insert : 1333762540 To insert : 1333762541 To insert : 1333762542 To insert : 1333762543 To insert : 1333762544 To insert : 1333762545 To insert : 1333762546 To insert : 1333762547 To insert : 1333762548 To insert : 1333762549 To insert : 1333762550 To insert : 1333762551 To insert : 1333762552 To insert : 1333762553 To insert : 1333762554 To insert : 1333762521 To insert : 1333762524 To insert : 1333762525 To insert : 1333762526 To insert : 1333762527 To insert : 1333762528 To insert : 1333762529 To insert : 1333762530 To insert : 1333762531 To insert : 1333762532 To insert : 1333762533 To insert : 1333762534 To insert : 1333762535 To insert : 1333762536 To insert : 1333762537 To insert : 1333762538 To insert : 1333762667 To insert : 1333762668 To insert : 1333762669 To insert : 1333762670 To insert : 1333762671 To insert : 1333762672 To insert : 1333762673 To insert : 1333762674 To insert : 1333762675 To insert : 1333762676 To insert : 1333762677 To insert : 1333762678 To insert : 1333762679 To insert : 1333762681 To insert : 1333762682 To insert : 1333762683 To insert : 1333762627 To insert : 1333762628 To insert : 1333762629 To insert : 1333762630 To insert : 1333762631 To insert : 1333762632 To insert : 1333762633 To insert : 1333762634 To insert : 1333762635 To insert : 1333762636 To insert : 1333762637 To insert : 1333762638 To insert : 1333762639 To insert : 1333762640 To insert : 1333762641 To insert : 1333762642 To insert : 1333762605 To insert : 1333762606 To insert : 1333762607 To insert : 1333762608 To insert : 1333762609 To insert : 1333762610 To insert : 1333762611 To insert : 1333762612 To insert : 1333762613 To insert : 1333762614 To insert : 1333762621 To insert : 1333762622 To insert : 1333762623 To insert : 1333762624 To insert : 1333762625 To insert : 1333762626 To insert : 1333762555 To insert : 1333762556 To insert : 1333762557 To insert : 1333762558 To insert : 1333762559 To insert : 1333762560 To insert : 1333762561 To insert : 1333762562 To insert : 1333762563 To insert : 1333762564 To insert : 1333762565 To insert : 1333762566 To insert : 1333762567 To insert : 1333762569 To insert : 1333762570 To insert : 1333762571 To insert : 1333762643 To insert : 1333762644 To insert : 1333762645 To insert : 1333762646 To insert : 1333762647 To insert : 1333762648 To insert : 1333762649 To insert : 1333762650 To insert : 1333762653 To insert : 1333762654 To insert : 1333762656 To insert : 1333762657 To insert : 1333762658 To insert : 1333762664 To insert : 1333762665 To insert : 1333762666 time to insert the descriptors : 27.791886568069458 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos SAVE THCL : begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 244 time used for this insertion : 0.022241592407226562 save missing photos in datou_result : time spend for datou_step_exec : 34.74792838096619 time spend to save output : 0.07134294509887695 total time spend for step 6 : 34.81927132606506 step7:ventilate_hashtags_in_portfolio Sat Feb 1 18:22:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 20143824 get user id for portfolio 20143824 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20143824 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Tetrapak','refus','flou','autre','Carton','mal_croppe','Papier','papier','pet_clair','pet_fonce','environnement','PET_clair','metal','PEHD','pehd','PET_fonce','carton','Film_plastique')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20143824 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Tetrapak','refus','flou','autre','Carton','mal_croppe','Papier','papier','pet_clair','pet_fonce','environnement','PET_clair','metal','PEHD','pehd','PET_fonce','carton','Film_plastique')) AND mptpi.`min_score`=0.6 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20143824 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Tetrapak','refus','flou','autre','Carton','mal_croppe','Papier','papier','pet_clair','pet_fonce','environnement','PET_clair','metal','PEHD','pehd','PET_fonce','carton','Film_plastique')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20143824 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Tetrapak','refus','flou','autre','Carton','mal_croppe','Papier','papier','pet_clair','pet_fonce','environnement','PET_clair','metal','PEHD','pehd','PET_fonce','carton','Film_plastique')) AND mptpi.`min_score`=0.6 To do lien utilise dans velours : https://www.fotonower.com/velours/20143978,20143979,20143980,20143981,20143994,20143983,20143985,20143989,20143993,20143988,20143990,20143992,20143995?tags=tetrapak,refus,flou,autre,carton,mal_croppe,papier,pet_clair,pet_fonce,environnement,metal,pehd,film_plastique&datou_id_consolidate=4235&port_consolidate=20143824 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1333757336, 1333757304] Looping around the photos to save general results len do output : 1 /20143824. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2537864') ('4234', '20143824', '1333757336', None, None, None, None, None, '2537864') ('4234', None, None, None, None, None, None, None, '2537864') ('4234', '20143824', '1333757304', None, None, None, None, None, '2537864') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.01155543327331543 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.251347064971924 time spend to save output : 0.011827945709228516 total time spend for step 7 : 2.2631750106811523 step8:final Sat Feb 1 18:22:33 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1333757336: ('0.2021491591470001',), 1333757304: ('0.2021491591470001',)} new output for save of step final : {1333757336: ('0.2021491591470001',), 1333757304: ('0.2021491591470001',)} [1333757336, 1333757304] Looping around the photos to save general results len do output : 2 /1333757336.Didn't retrieve data . /1333757304.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2537864') ('4234', '20143824', '1333757336', None, None, None, None, None, '2537864') ('4234', None, None, None, None, None, None, None, '2537864') ('4234', '20143824', '1333757304', None, None, None, None, None, '2537864') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.011094093322753906 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.1270463466644287 time spend to save output : 0.011412382125854492 total time spend for step 8 : 0.1384587287902832 step9:velours_tree Sat Feb 1 18:22:33 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure list_portfolios : 20143978,20143979,20143980,20143981,20143994,20143983,20143985,20143989,20143993,20143988,20143990,20143992,20143995 photo desc type : 5561 - Retrieving photos to tag... query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20143978 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20143979 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20143980 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20143981 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20143994 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20143983 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20143985 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20143989 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20143993 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20143988 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20143990 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20143992 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20143995 ORDER BY ph.size desc - Loading descriptors... Size : 2048 len(descriptors) : 156 Compute structured hierarchical clustering... ward : AgglomerativeClustering(n_clusters=6) ward.labels_ : [4 0 5 3 3 1 1 1 1 3 1 1 0 0 0 0 0 1 1 1 1 1 0 1 1 0 0 0 0 1 1 0 0 4 0 5 4 3 4 3 1 3 3 0 5 3 3 3 3 3 1 1 1 1 3 0 1 0 0 0 0 1 1 1 1 1 1 0 0 0 1 1 1 1 3 0 1 4 0 0 5 4 0 3 3 1 5 3 1 3 2 4 3 3 3 4 5 4 1 2 0 2 1 2 1 3 3 0 1 3 3 3 4 5 4 4 1 2 2 0 3 0 1 4 1 4 2 0 0 0 2 0 4 1 1 3 1 4 0 4 4 1 4 4 2 0 2 4 1 0 5 4 4 4 0 4] Elapsed time: 0.02875375747680664 graph_id used : 75127 - Beta version, working pretty good on 11-5-16 ! https://www.fotonower.com/velours/20143978,20143979,20143980,20143981,20143994,20143983,20143985,20143989,20143993,20143988,20143990,20143992,20143995?tags=tetrapak,refus,flou,autre,carton,mal_croppe,papier,pet_clair,pet_fonce,environnement,metal,pehd,film_plastique&datou_id_consolidate=4235&port_consolidate=20143824&tree_id=75127 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : velours_tree we use saveGeneral [1333757336, 1333757304] Looping around the photos to save general results len do output : 1 /20143824Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2537864') ('4234', '20143824', '1333757336', None, None, None, None, None, '2537864') ('4234', None, None, None, None, None, None, None, '2537864') ('4234', '20143824', '1333757304', None, None, None, None, None, '2537864') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.012057065963745117 save_final save missing photos in datou_result : time spend for datou_step_exec : 9.340936660766602 time spend to save output : 0.012384176254272461 total time spend for step 9 : 9.353320837020874 step10:send_mail_cod Sat Feb 1 18:22:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Qualipapia_P20143824_01-02-2025_18_22_42.pdf 20143960 imagette201439601738430562 20143961 imagette201439611738430562 20143962 imagette201439621738430562 20143963 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette201439631738430562 20143976 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette201439761738430563 20143965 imagette201439651738430565 20143967 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette201439671738430565 20143971 change filename to text .change filename to text .change filename to text .imagette201439711738430566 20143975 imagette201439751738430566 20143972 change filename to text .change filename to text .imagette201439721738430566 20143974 imagette201439741738430566 20143977 imagette201439771738430566 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20143824 and hashtag_type = 4230 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20143978,20143979,20143980,20143981,20143994,20143983,20143985,20143989,20143993,20143988,20143990,20143992,20143995?tags=tetrapak,refus,flou,autre,carton,mal_croppe,papier,pet_clair,pet_fonce,environnement,metal,pehd,film_plastique&datou_id_consolidate=4235&port_consolidate=20143824&tree_id=75127 your option no_mail is active, we will not send the real mail to your client args[1333757336] : ((1333757336, -7.980425247250039, 492609224), (1333757336, -0.5397818318358419, 501862349), '0.2021491591470001') We are sending mail with results at cod@fotonower.com args[1333757304] : ((1333757304, -8.011329491921765, 492609224), (1333757304, -0.504939296238048, 501862349), '0.2021491591470001') We are sending mail with results at cod@fotonower.com refus_total : 0.2021491591470001 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20143824 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20143824_01-02-2025_18_22_42.pdf results_Qualipapia_P20143824_01-02-2025_18_22_42.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20143824_01-02-2025_18_22_42.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4234','20143824','results_Qualipapia_P20143824_01-02-2025_18_22_42.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20143824_01-02-2025_18_22_42.pdf','pdf','','0.14','0.2021491591470001') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1333757336, 1333757304] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2537864') ('4234', '20143824', '1333757336', None, None, None, None, None, '2537864') ('4234', None, None, None, None, None, None, None, '2537864') ('4234', '20143824', '1333757304', None, None, None, None, None, '2537864') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 2 time used for this insertion : 0.013119697570800781 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.599803924560547 time spend to save output : 0.013298273086547852 total time spend for step 10 : 4.613102197647095 step11:split_time_score Sat Feb 1 18:22:47 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 3442, 'mtr_user_id': 31, 'name': 'classifieur_2camions_valcor_021122_v1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'deux_camions,camion_droite,camion_gauche,pas_de_camion', 'svm_portfolios_learning': '7659379,7659034,7657685,7657114', 'photo_hashtag_type': 4458, 'photo_desc_type': 5723, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107760533,2107760534,2107760535,2107760536'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('15', 2),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 01022025 20143824 Nombre de photos uploadées : 2 / 23040 (0%) 01022025 20143824 Nombre de photos taguées (types de déchets): 2 / 2 (100%) 01022025 20143824 Nombre de photos taguées (volume) : 0 / 2 (0%) elapsed_time : load_data_split_time_score 1.6689300537109375e-06 elapsed_time : order_list_meta_photo_and_scores 5.245208740234375e-06 ?? elapsed_time : fill_and_build_computed_from_old_data 0.0001881122589111328 elapsed_time : insert_dashboard_record_day_entry 0.024699926376342773 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140596 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140597 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140598 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140599 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140600 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140601 order by id desc limit 1 Qualite : 0.2309767904458083 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20140602_01-02-2025_13_40_01.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140602 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20140602 AND mptpi.`type`=4230 To do Qualite : 0.22262298814595152 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20140603_01-02-2025_13_36_33.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140603 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20140603 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140832 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140833 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140834 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140835 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140836 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140837 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140838 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140839 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140840 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140841 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140842 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140843 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140844 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140845 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140846 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140847 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140848 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140849 order by id desc limit 1 Qualite : 0.20057559398632832 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20140850_01-02-2025_14_10_13.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140850 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20140850 AND mptpi.`type`=4230 To do Qualite : 0.18431451539642255 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20140851_01-02-2025_13_35_17.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140851 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20140851 AND mptpi.`type`=4230 To do Qualite : 0.19593863158258523 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20140852_01-02-2025_13_53_55.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20140852 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20140852 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141296 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141297 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141298 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141299 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141300 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141301 order by id desc limit 1 Qualite : 0.20015635039197005 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20141302_01-02-2025_14_32_21.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141302 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20141302 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141657 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141658 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141659 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141660 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141661 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141662 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141663 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141664 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141665 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141666 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141667 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141668 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141669 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141670 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141671 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141672 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141673 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141674 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141675 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141676 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141677 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141678 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141679 order by id desc limit 1 Qualite : 0.20827029105652828 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20141680_01-02-2025_15_08_13.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141680 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20141680 AND mptpi.`type`=4230 To do Qualite : 0.1946503779440649 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20141681_01-02-2025_14_44_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20141681 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20141681 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142049 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142050 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142051 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142052 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142053 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142054 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142055 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142056 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142057 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142058 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142059 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142060 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142061 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142062 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142063 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142064 order by id desc limit 1 Qualite : 0.12794507052083104 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20142065_01-02-2025_15_33_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142065 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20142065 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142066 order by id desc limit 1 Qualite : 0.19287199063881355 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20142067_01-02-2025_15_18_52.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142067 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20142067 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142451 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142452 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142453 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142454 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142455 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142456 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142457 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142458 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142459 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142460 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142461 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142462 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142463 order by id desc limit 1 Qualite : 0.18838141105926856 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20142464_01-02-2025_16_02_33.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142464 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20142464 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142465 order by id desc limit 1 Qualite : 0.22140410901851382 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20142466_01-02-2025_15_47_57.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142466 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20142466 AND mptpi.`type`=4230 To do Qualite : 0.24002910245386597 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20142467_01-02-2025_15_40_34.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142467 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20142467 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142783 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142784 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142785 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142786 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142787 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142788 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142789 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142790 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142791 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142792 order by id desc limit 1 Qualite : 0.3079498429232401 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20142793_01-02-2025_16_32_12.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142793 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20142793 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142794 order by id desc limit 1 Qualite : 0.19429584143844753 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20142795_01-02-2025_16_12_46.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20142795 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20142795 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143051 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143052 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143053 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143054 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143055 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143056 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143057 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143058 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143059 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143060 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143061 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143062 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143063 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143064 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143065 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143066 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143067 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143068 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143069 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143070 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143071 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143072 order by id desc limit 1 Qualite : 0.2019070967415384 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20143073_01-02-2025_17_06_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143073 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20143073 AND mptpi.`type`=4230 To do Qualite : 0.22595934725209532 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20143074_01-02-2025_16_46_55.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143074 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20143074 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143392 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143393 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143394 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143395 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143396 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143397 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143398 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143399 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143400 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143401 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143402 order by id desc limit 1 Qualite : 0.23912616429849357 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20143403_01-02-2025_17_33_02.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143403 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20143403 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143404 order by id desc limit 1 Qualite : 0.2207605871752271 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20143405_01-02-2025_17_10_53.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143405 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20143405 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143406 order by id desc limit 1 Qualite : 0.2229691740437331 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20143407_01-02-2025_17_18_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143407 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20143407 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143697 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143698 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143699 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143700 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143701 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143702 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143703 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143704 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143705 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143706 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143707 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143708 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143709 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143710 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143711 order by id desc limit 1 Qualite : 0.19745974180902578 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20143712_01-02-2025_17_47_16.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143712 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20143712 AND mptpi.`type`=4230 To do Qualite : 0.20609688696404269 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20143713_01-02-2025_18_07_00.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143713 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20143713 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143823 order by id desc limit 1 Qualite : 0.2021491591470001 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20143824_01-02-2025_18_22_42.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143824 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20143824 AND mptpi.`type`=4230 To do Qualite : 0.18827936840275175 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20143825_01-02-2025_18_16_56.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143825 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20143825 AND mptpi.`type`=4230 To do Qualite : 0.18073680563575475 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20143827_01-02-2025_18_11_42.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20143827 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20143827 AND mptpi.`type`=4230 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'01022025': {'nb_upload': 2, 'nb_taggue_class': 2, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1333757336, 1333757304] Looping around the photos to save general results len do output : 1 /20143824Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2537864') ('4234', '20143824', '1333757336', None, None, None, None, None, '2537864') ('4234', None, None, None, None, None, None, None, '2537864') ('4234', '20143824', '1333757304', None, None, None, None, None, '2537864') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.012834787368774414 save_final save missing photos in datou_result : time spend for datou_step_exec : 109.10561895370483 time spend to save output : 0.01303410530090332 total time spend for step 11 : 109.11865305900574 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 1 len(input) + len(total_photo_id_missing) : 2 set_done_treatment 33.39user 26.71system 4:12.29elapsed 23%CPU (0avgtext+0avgdata 2500364maxresident)k 2430424inputs+11992outputs (8088major+2185057minor)pagefaults 0swaps