python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 4234 -l 5 ' -s datou_current_4234 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 753391 load datou : 4234 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load THCL from format json or kwargs add thcl : 3237 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig add pdt : 5561 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4234, datou_cur_ids : ['2560851'] with mtr_portfolio_ids : ['20313900'] and first list_photo_ids : [] new path : /proc/753391/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, brightness, blur_detection, rle_unique_nms_with_priority, crop_condition, thcl, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , BFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 2 ; length of list_pids : 2 ; length of list_args : 2 time to download the photos : 0.3120238780975342 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Fri Feb 7 00:20:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 6469 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-07 00:20:32.501216: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-07 00:20:32.527162: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-07 00:20:32.528889: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f32b4000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-07 00:20:32.528945: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-07 00:20:32.531915: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-07 00:20:32.801269: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x34cd1be0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-07 00:20:32.801334: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-07 00:20:32.802365: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-07 00:20:32.802771: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-07 00:20:32.805080: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-07 00:20:32.807630: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-07 00:20:32.808095: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-07 00:20:32.810676: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-07 00:20:32.811976: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-07 00:20:32.817492: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-07 00:20:32.818978: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-07 00:20:32.819106: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-07 00:20:32.819834: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-07 00:20:32.819852: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-07 00:20:32.819862: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-07 00:20:32.821057: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 5938 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-07 00:20:33.129240: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-07 00:20:33.129365: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-07 00:20:33.129384: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-07 00:20:33.129400: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-07 00:20:33.129416: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-07 00:20:33.129431: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-07 00:20:33.129445: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-07 00:20:33.129460: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-07 00:20:33.130556: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-07 00:20:33.131833: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-07 00:20:33.131889: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-07 00:20:33.131908: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-07 00:20:33.131923: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-07 00:20:33.131936: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-07 00:20:33.131950: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-07 00:20:33.131964: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-07 00:20:33.131979: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-07 00:20:33.133068: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-07 00:20:33.133112: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-07 00:20:33.133122: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-07 00:20:33.133130: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-07 00:20:33.134251: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 5938 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-07 00:20:42.564194: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-07 00:20:42.784225: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 2 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -116.90703 max: 149.15078 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 7.367134094238281e-05 nb_pixel_total : 372 time to create 1 rle with old method : 0.0007624626159667969 length of segment : 38 time for calcul the mask position with numpy : 6.0558319091796875e-05 nb_pixel_total : 977 time to create 1 rle with old method : 0.0017664432525634766 length of segment : 41 time for calcul the mask position with numpy : 4.220008850097656e-05 nb_pixel_total : 101 time to create 1 rle with old method : 0.00025463104248046875 length of segment : 24 time for calcul the mask position with numpy : 5.650520324707031e-05 nb_pixel_total : 44 time to create 1 rle with old method : 0.00016641616821289062 length of segment : 10 time for calcul the mask position with numpy : 0.0004334449768066406 nb_pixel_total : 10719 time to create 1 rle with old method : 0.01788806915283203 length of segment : 201 time for calcul the mask position with numpy : 4.172325134277344e-05 nb_pixel_total : 39 time to create 1 rle with old method : 0.00010704994201660156 length of segment : 14 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -116.91875 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 6.890296936035156e-05 nb_pixel_total : 2461 time to create 1 rle with old method : 0.0031173229217529297 length of segment : 53 time for calcul the mask position with numpy : 3.790855407714844e-05 nb_pixel_total : 263 time to create 1 rle with old method : 0.00036215782165527344 length of segment : 30 time for calcul the mask position with numpy : 5.3882598876953125e-05 nb_pixel_total : 1139 time to create 1 rle with old method : 0.0013887882232666016 length of segment : 46 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -122.22344 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 7.677078247070312e-05 nb_pixel_total : 1328 time to create 1 rle with old method : 0.00604248046875 length of segment : 69 time for calcul the mask position with numpy : 9.703636169433594e-05 nb_pixel_total : 3276 time to create 1 rle with old method : 0.004095792770385742 length of segment : 55 time for calcul the mask position with numpy : 0.00011706352233886719 nb_pixel_total : 2701 time to create 1 rle with old method : 0.0034410953521728516 length of segment : 100 time for calcul the mask position with numpy : 0.00017070770263671875 nb_pixel_total : 5202 time to create 1 rle with old method : 0.008884191513061523 length of segment : 101 time for calcul the mask position with numpy : 0.0001347064971923828 nb_pixel_total : 4285 time to create 1 rle with old method : 0.008109092712402344 length of segment : 57 time for calcul the mask position with numpy : 0.00010395050048828125 nb_pixel_total : 1600 time to create 1 rle with old method : 0.00308990478515625 length of segment : 63 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -109.50859 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 6.771087646484375e-05 nb_pixel_total : 415 time to create 1 rle with old method : 0.0008594989776611328 length of segment : 23 time for calcul the mask position with numpy : 0.000194549560546875 nb_pixel_total : 9554 time to create 1 rle with old method : 0.011761665344238281 length of segment : 131 time for calcul the mask position with numpy : 3.8623809814453125e-05 nb_pixel_total : 189 time to create 1 rle with old method : 0.0002796649932861328 length of segment : 16 time for calcul the mask position with numpy : 4.100799560546875e-05 nb_pixel_total : 486 time to create 1 rle with old method : 0.0007376670837402344 length of segment : 24 time for calcul the mask position with numpy : 0.0002503395080566406 nb_pixel_total : 8315 time to create 1 rle with old method : 0.011164665222167969 length of segment : 150 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 141.82656 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 5.412101745605469e-05 nb_pixel_total : 490 time to create 1 rle with old method : 0.0007047653198242188 length of segment : 18 time for calcul the mask position with numpy : 3.409385681152344e-05 nb_pixel_total : 97 time to create 1 rle with old method : 0.00013780593872070312 length of segment : 29 time for calcul the mask position with numpy : 0.0006787776947021484 nb_pixel_total : 55262 time to create 1 rle with old method : 0.07785248756408691 length of segment : 277 time for calcul the mask position with numpy : 9.465217590332031e-05 nb_pixel_total : 1611 time to create 1 rle with old method : 0.002001047134399414 length of segment : 105 time for calcul the mask position with numpy : 0.0006811618804931641 nb_pixel_total : 64376 time to create 1 rle with old method : 0.09716916084289551 length of segment : 288 Processing 1 images image shape: (400, 400, 3) min: 20.00000 max: 203.00000 molded_images shape: (1, 640, 640, 3) min: -83.32891 max: 72.36406 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -106.00469 max: 150.70156 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 14 time for calcul the mask position with numpy : 6.771087646484375e-05 nb_pixel_total : 199 time to create 1 rle with old method : 0.0003056526184082031 length of segment : 14 time for calcul the mask position with numpy : 4.315376281738281e-05 nb_pixel_total : 501 time to create 1 rle with old method : 0.0006961822509765625 length of segment : 12 time for calcul the mask position with numpy : 3.24249267578125e-05 nb_pixel_total : 54 time to create 1 rle with old method : 0.0001125335693359375 length of segment : 7 time for calcul the mask position with numpy : 3.0994415283203125e-05 nb_pixel_total : 66 time to create 1 rle with old method : 0.00011348724365234375 length of segment : 10 time for calcul the mask position with numpy : 3.743171691894531e-05 nb_pixel_total : 365 time to create 1 rle with old method : 0.0005357265472412109 length of segment : 33 time for calcul the mask position with numpy : 3.933906555175781e-05 nb_pixel_total : 125 time to create 1 rle with old method : 0.0002105236053466797 length of segment : 15 time for calcul the mask position with numpy : 3.1948089599609375e-05 nb_pixel_total : 175 time to create 1 rle with old method : 0.0002627372741699219 length of segment : 14 time for calcul the mask position with numpy : 5.2928924560546875e-05 nb_pixel_total : 1046 time to create 1 rle with old method : 0.00147247314453125 length of segment : 31 time for calcul the mask position with numpy : 0.00010943412780761719 nb_pixel_total : 1026 time to create 1 rle with old method : 0.0014600753784179688 length of segment : 66 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 80 time to create 1 rle with old method : 0.00021409988403320312 length of segment : 14 time for calcul the mask position with numpy : 3.719329833984375e-05 nb_pixel_total : 118 time to create 1 rle with old method : 0.00020265579223632812 length of segment : 14 time for calcul the mask position with numpy : 5.245208740234375e-05 nb_pixel_total : 999 time to create 1 rle with old method : 0.0013508796691894531 length of segment : 58 time for calcul the mask position with numpy : 9.489059448242188e-05 nb_pixel_total : 1088 time to create 1 rle with old method : 0.002590656280517578 length of segment : 43 time for calcul the mask position with numpy : 5.1021575927734375e-05 nb_pixel_total : 259 time to create 1 rle with old method : 0.0004982948303222656 length of segment : 21 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 12 time for calcul the mask position with numpy : 6.818771362304688e-05 nb_pixel_total : 598 time to create 1 rle with old method : 0.0008575916290283203 length of segment : 21 time for calcul the mask position with numpy : 6.67572021484375e-05 nb_pixel_total : 666 time to create 1 rle with old method : 0.0009109973907470703 length of segment : 30 time for calcul the mask position with numpy : 5.7220458984375e-05 nb_pixel_total : 707 time to create 1 rle with old method : 0.0009267330169677734 length of segment : 38 time for calcul the mask position with numpy : 5.125999450683594e-05 nb_pixel_total : 213 time to create 1 rle with old method : 0.0004315376281738281 length of segment : 12 time for calcul the mask position with numpy : 3.933906555175781e-05 nb_pixel_total : 118 time to create 1 rle with old method : 0.00021123886108398438 length of segment : 14 time for calcul the mask position with numpy : 0.0001227855682373047 nb_pixel_total : 4099 time to create 1 rle with old method : 0.005018472671508789 length of segment : 97 time for calcul the mask position with numpy : 0.00010561943054199219 nb_pixel_total : 1677 time to create 1 rle with old method : 0.0029344558715820312 length of segment : 32 time for calcul the mask position with numpy : 6.175041198730469e-05 nb_pixel_total : 322 time to create 1 rle with old method : 0.0005850791931152344 length of segment : 35 time for calcul the mask position with numpy : 5.245208740234375e-05 nb_pixel_total : 251 time to create 1 rle with old method : 0.0004887580871582031 length of segment : 24 time for calcul the mask position with numpy : 8.654594421386719e-05 nb_pixel_total : 1642 time to create 1 rle with old method : 0.0029897689819335938 length of segment : 32 time for calcul the mask position with numpy : 8.726119995117188e-05 nb_pixel_total : 1740 time to create 1 rle with old method : 0.0029735565185546875 length of segment : 53 time for calcul the mask position with numpy : 7.796287536621094e-05 nb_pixel_total : 1461 time to create 1 rle with old method : 0.002757549285888672 length of segment : 30 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -119.63750 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 12 time for calcul the mask position with numpy : 0.00013899803161621094 nb_pixel_total : 3502 time to create 1 rle with old method : 0.005144834518432617 length of segment : 101 time for calcul the mask position with numpy : 0.0001742839813232422 nb_pixel_total : 2467 time to create 1 rle with old method : 0.004213094711303711 length of segment : 89 time for calcul the mask position with numpy : 5.984306335449219e-05 nb_pixel_total : 118 time to create 1 rle with old method : 0.0002048015594482422 length of segment : 16 time for calcul the mask position with numpy : 3.6716461181640625e-05 nb_pixel_total : 20 time to create 1 rle with old method : 6.604194641113281e-05 length of segment : 6 time for calcul the mask position with numpy : 0.00029850006103515625 nb_pixel_total : 10671 time to create 1 rle with old method : 0.013092041015625 length of segment : 304 time for calcul the mask position with numpy : 9.083747863769531e-05 nb_pixel_total : 872 time to create 1 rle with old method : 0.0014414787292480469 length of segment : 42 time for calcul the mask position with numpy : 0.00019931793212890625 nb_pixel_total : 1848 time to create 1 rle with old method : 0.002794981002807617 length of segment : 86 time for calcul the mask position with numpy : 4.673004150390625e-05 nb_pixel_total : 328 time to create 1 rle with old method : 0.0005562305450439453 length of segment : 23 time for calcul the mask position with numpy : 6.270408630371094e-05 nb_pixel_total : 1380 time to create 1 rle with old method : 0.0019259452819824219 length of segment : 44 time for calcul the mask position with numpy : 0.00022220611572265625 nb_pixel_total : 6152 time to create 1 rle with old method : 0.009928226470947266 length of segment : 227 time for calcul the mask position with numpy : 7.414817810058594e-05 nb_pixel_total : 1288 time to create 1 rle with old method : 0.0017409324645996094 length of segment : 40 time for calcul the mask position with numpy : 5.435943603515625e-05 nb_pixel_total : 867 time to create 1 rle with old method : 0.0011625289916992188 length of segment : 37 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 149.77969 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 11 time for calcul the mask position with numpy : 4.649162292480469e-05 nb_pixel_total : 470 time to create 1 rle with old method : 0.0006694793701171875 length of segment : 36 time for calcul the mask position with numpy : 0.0002155303955078125 nb_pixel_total : 6345 time to create 1 rle with old method : 0.00817108154296875 length of segment : 242 time for calcul the mask position with numpy : 0.0001933574676513672 nb_pixel_total : 2184 time to create 1 rle with old method : 0.002953052520751953 length of segment : 179 time for calcul the mask position with numpy : 6.008148193359375e-05 nb_pixel_total : 242 time to create 1 rle with old method : 0.0003864765167236328 length of segment : 22 time for calcul the mask position with numpy : 4.38690185546875e-05 nb_pixel_total : 376 time to create 1 rle with old method : 0.0005326271057128906 length of segment : 39 time for calcul the mask position with numpy : 9.1552734375e-05 nb_pixel_total : 1071 time to create 1 rle with old method : 0.001481771469116211 length of segment : 125 time for calcul the mask position with numpy : 0.00010180473327636719 nb_pixel_total : 1730 time to create 1 rle with old method : 0.002177000045776367 length of segment : 83 time for calcul the mask position with numpy : 7.82012939453125e-05 nb_pixel_total : 375 time to create 1 rle with old method : 0.0006780624389648438 length of segment : 55 time for calcul the mask position with numpy : 3.600120544433594e-05 nb_pixel_total : 355 time to create 1 rle with old method : 0.0005033016204833984 length of segment : 30 time for calcul the mask position with numpy : 5.0067901611328125e-05 nb_pixel_total : 400 time to create 1 rle with old method : 0.0006301403045654297 length of segment : 39 time for calcul the mask position with numpy : 8.821487426757812e-05 nb_pixel_total : 1803 time to create 1 rle with old method : 0.002642393112182617 length of segment : 87 Processing 1 images image shape: (280, 400, 3) min: 8.00000 max: 209.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 80.86172 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 6.270408630371094e-05 nb_pixel_total : 476 time to create 1 rle with old method : 0.0008826255798339844 length of segment : 54 time for calcul the mask position with numpy : 0.0012159347534179688 nb_pixel_total : 106557 time to create 1 rle with old method : 0.13721513748168945 length of segment : 286 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.15469 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 15 time for calcul the mask position with numpy : 8.821487426757812e-05 nb_pixel_total : 1734 time to create 1 rle with old method : 0.0030379295349121094 length of segment : 43 time for calcul the mask position with numpy : 7.700920104980469e-05 nb_pixel_total : 1497 time to create 1 rle with old method : 0.002773284912109375 length of segment : 48 time for calcul the mask position with numpy : 0.00021266937255859375 nb_pixel_total : 7687 time to create 1 rle with old method : 0.013121604919433594 length of segment : 102 time for calcul the mask position with numpy : 6.937980651855469e-05 nb_pixel_total : 415 time to create 1 rle with old method : 0.000675201416015625 length of segment : 30 time for calcul the mask position with numpy : 5.888938903808594e-05 nb_pixel_total : 611 time to create 1 rle with old method : 0.0010116100311279297 length of segment : 26 time for calcul the mask position with numpy : 4.3392181396484375e-05 nb_pixel_total : 392 time to create 1 rle with old method : 0.0006852149963378906 length of segment : 24 time for calcul the mask position with numpy : 4.553794860839844e-05 nb_pixel_total : 758 time to create 1 rle with old method : 0.0009875297546386719 length of segment : 30 time for calcul the mask position with numpy : 3.504753112792969e-05 nb_pixel_total : 187 time to create 1 rle with old method : 0.0002872943878173828 length of segment : 16 time for calcul the mask position with numpy : 3.695487976074219e-05 nb_pixel_total : 295 time to create 1 rle with old method : 0.00041747093200683594 length of segment : 22 time for calcul the mask position with numpy : 4.6253204345703125e-05 nb_pixel_total : 460 time to create 1 rle with old method : 0.0006842613220214844 length of segment : 29 time for calcul the mask position with numpy : 4.363059997558594e-05 nb_pixel_total : 131 time to create 1 rle with old method : 0.0002257823944091797 length of segment : 16 time for calcul the mask position with numpy : 4.315376281738281e-05 nb_pixel_total : 180 time to create 1 rle with old method : 0.0002818107604980469 length of segment : 14 time for calcul the mask position with numpy : 6.031990051269531e-05 nb_pixel_total : 226 time to create 1 rle with old method : 0.00047016143798828125 length of segment : 23 time for calcul the mask position with numpy : 4.982948303222656e-05 nb_pixel_total : 300 time to create 1 rle with old method : 0.0004627704620361328 length of segment : 18 time for calcul the mask position with numpy : 5.364418029785156e-05 nb_pixel_total : 806 time to create 1 rle with old method : 0.001085042953491211 length of segment : 31 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.35391 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 25 time for calcul the mask position with numpy : 7.295608520507812e-05 nb_pixel_total : 295 time to create 1 rle with old method : 0.0004875659942626953 length of segment : 21 time for calcul the mask position with numpy : 7.891654968261719e-05 nb_pixel_total : 678 time to create 1 rle with old method : 0.0010373592376708984 length of segment : 34 time for calcul the mask position with numpy : 5.7697296142578125e-05 nb_pixel_total : 317 time to create 1 rle with old method : 0.0005502700805664062 length of segment : 20 time for calcul the mask position with numpy : 5.817413330078125e-05 nb_pixel_total : 139 time to create 1 rle with old method : 0.00027632713317871094 length of segment : 15 time for calcul the mask position with numpy : 5.435943603515625e-05 nb_pixel_total : 365 time to create 1 rle with old method : 0.0005602836608886719 length of segment : 23 time for calcul the mask position with numpy : 4.6253204345703125e-05 nb_pixel_total : 170 time to create 1 rle with old method : 0.000278472900390625 length of segment : 17 time for calcul the mask position with numpy : 4.601478576660156e-05 nb_pixel_total : 201 time to create 1 rle with old method : 0.00034689903259277344 length of segment : 16 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 189 time to create 1 rle with old method : 0.0003757476806640625 length of segment : 11 time for calcul the mask position with numpy : 4.100799560546875e-05 nb_pixel_total : 253 time to create 1 rle with old method : 0.0004413127899169922 length of segment : 34 time for calcul the mask position with numpy : 4.410743713378906e-05 nb_pixel_total : 282 time to create 1 rle with old method : 0.0004162788391113281 length of segment : 24 time for calcul the mask position with numpy : 3.504753112792969e-05 nb_pixel_total : 201 time to create 1 rle with old method : 0.00033783912658691406 length of segment : 17 time for calcul the mask position with numpy : 3.600120544433594e-05 nb_pixel_total : 105 time to create 1 rle with old method : 0.00021219253540039062 length of segment : 10 time for calcul the mask position with numpy : 4.9591064453125e-05 nb_pixel_total : 527 time to create 1 rle with old method : 0.0007755756378173828 length of segment : 52 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 161 time to create 1 rle with old method : 0.0002846717834472656 length of segment : 12 time for calcul the mask position with numpy : 4.696846008300781e-05 nb_pixel_total : 546 time to create 1 rle with old method : 0.000789642333984375 length of segment : 33 time for calcul the mask position with numpy : 6.818771362304688e-05 nb_pixel_total : 265 time to create 1 rle with old method : 0.0007543563842773438 length of segment : 21 time for calcul the mask position with numpy : 6.341934204101562e-05 nb_pixel_total : 77 time to create 1 rle with old method : 0.0002999305725097656 length of segment : 7 time for calcul the mask position with numpy : 5.841255187988281e-05 nb_pixel_total : 361 time to create 1 rle with old method : 0.0005371570587158203 length of segment : 36 time for calcul the mask position with numpy : 4.649162292480469e-05 nb_pixel_total : 130 time to create 1 rle with old method : 0.00022602081298828125 length of segment : 14 time for calcul the mask position with numpy : 7.295608520507812e-05 nb_pixel_total : 1054 time to create 1 rle with old method : 0.0014941692352294922 length of segment : 45 time for calcul the mask position with numpy : 4.267692565917969e-05 nb_pixel_total : 61 time to create 1 rle with old method : 0.0001800060272216797 length of segment : 7 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 517 time to create 1 rle with old method : 0.0007634162902832031 length of segment : 43 time for calcul the mask position with numpy : 3.647804260253906e-05 nb_pixel_total : 243 time to create 1 rle with old method : 0.0003795623779296875 length of segment : 18 time for calcul the mask position with numpy : 3.4809112548828125e-05 nb_pixel_total : 111 time to create 1 rle with old method : 0.00019407272338867188 length of segment : 13 time for calcul the mask position with numpy : 4.2438507080078125e-05 nb_pixel_total : 407 time to create 1 rle with old method : 0.0007305145263671875 length of segment : 18 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.00234 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 13 time for calcul the mask position with numpy : 5.936622619628906e-05 nb_pixel_total : 1533 time to create 1 rle with old method : 0.0019409656524658203 length of segment : 34 time for calcul the mask position with numpy : 4.744529724121094e-05 nb_pixel_total : 257 time to create 1 rle with old method : 0.00040078163146972656 length of segment : 22 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 249 time to create 1 rle with old method : 0.00042057037353515625 length of segment : 19 time for calcul the mask position with numpy : 7.033348083496094e-05 nb_pixel_total : 767 time to create 1 rle with old method : 0.0010094642639160156 length of segment : 67 time for calcul the mask position with numpy : 3.4809112548828125e-05 nb_pixel_total : 105 time to create 1 rle with old method : 0.0001761913299560547 length of segment : 12 time for calcul the mask position with numpy : 3.9577484130859375e-05 nb_pixel_total : 433 time to create 1 rle with old method : 0.0006093978881835938 length of segment : 19 time for calcul the mask position with numpy : 6.198883056640625e-05 nb_pixel_total : 1055 time to create 1 rle with old method : 0.0013670921325683594 length of segment : 48 time for calcul the mask position with numpy : 7.319450378417969e-05 nb_pixel_total : 1547 time to create 1 rle with old method : 0.0020627975463867188 length of segment : 73 time for calcul the mask position with numpy : 5.626678466796875e-05 nb_pixel_total : 1058 time to create 1 rle with old method : 0.0016245841979980469 length of segment : 36 time for calcul the mask position with numpy : 4.839897155761719e-05 nb_pixel_total : 827 time to create 1 rle with old method : 0.0013496875762939453 length of segment : 39 time for calcul the mask position with numpy : 5.1975250244140625e-05 nb_pixel_total : 110 time to create 1 rle with old method : 0.0003218650817871094 length of segment : 8 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 226 time to create 1 rle with old method : 0.0003845691680908203 length of segment : 25 time for calcul the mask position with numpy : 3.790855407714844e-05 nb_pixel_total : 170 time to create 1 rle with old method : 0.0003108978271484375 length of segment : 19 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 134.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 5.602836608886719e-05 nb_pixel_total : 237 time to create 1 rle with old method : 0.0004172325134277344 length of segment : 58 time for calcul the mask position with numpy : 3.4809112548828125e-05 nb_pixel_total : 50 time to create 1 rle with old method : 9.584426879882812e-05 length of segment : 9 time for calcul the mask position with numpy : 3.5762786865234375e-05 nb_pixel_total : 419 time to create 1 rle with old method : 0.0006220340728759766 length of segment : 23 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.52422 max: 148.43984 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 8.940696716308594e-05 nb_pixel_total : 333 time to create 1 rle with old method : 0.00046062469482421875 length of segment : 30 time for calcul the mask position with numpy : 5.7220458984375e-05 nb_pixel_total : 893 time to create 1 rle with old method : 0.0012362003326416016 length of segment : 36 time for calcul the mask position with numpy : 3.981590270996094e-05 nb_pixel_total : 62 time to create 1 rle with old method : 0.00014448165893554688 length of segment : 19 time for calcul the mask position with numpy : 3.457069396972656e-05 nb_pixel_total : 20 time to create 1 rle with old method : 5.0067901611328125e-05 length of segment : 6 time for calcul the mask position with numpy : 4.482269287109375e-05 nb_pixel_total : 853 time to create 1 rle with old method : 0.0011172294616699219 length of segment : 34 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -116.38750 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 6.961822509765625e-05 nb_pixel_total : 2763 time to create 1 rle with old method : 0.003345012664794922 length of segment : 57 time for calcul the mask position with numpy : 6.651878356933594e-05 nb_pixel_total : 1710 time to create 1 rle with old method : 0.002039194107055664 length of segment : 81 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -122.89141 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 3.719329833984375e-05 nb_pixel_total : 41 time to create 1 rle with old method : 8.654594421386719e-05 length of segment : 11 time for calcul the mask position with numpy : 8.130073547363281e-05 nb_pixel_total : 438 time to create 1 rle with old method : 0.0006165504455566406 length of segment : 43 time for calcul the mask position with numpy : 0.00010633468627929688 nb_pixel_total : 5089 time to create 1 rle with old method : 0.0061016082763671875 length of segment : 86 time for calcul the mask position with numpy : 9.489059448242188e-05 nb_pixel_total : 1267 time to create 1 rle with old method : 0.0018880367279052734 length of segment : 86 time for calcul the mask position with numpy : 5.626678466796875e-05 nb_pixel_total : 1368 time to create 1 rle with old method : 0.0017867088317871094 length of segment : 61 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -112.71953 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 0.0001614093780517578 nb_pixel_total : 11292 time to create 1 rle with old method : 0.013199806213378906 length of segment : 111 time for calcul the mask position with numpy : 3.528594970703125e-05 nb_pixel_total : 211 time to create 1 rle with old method : 0.0003192424774169922 length of segment : 18 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 147.13516 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 4.076957702636719e-05 nb_pixel_total : 472 time to create 1 rle with old method : 0.0006649494171142578 length of segment : 20 time for calcul the mask position with numpy : 3.075599670410156e-05 nb_pixel_total : 96 time to create 1 rle with old method : 0.00013828277587890625 length of segment : 26 time for calcul the mask position with numpy : 0.0006017684936523438 nb_pixel_total : 54468 time to create 1 rle with old method : 0.061582326889038086 length of segment : 288 time for calcul the mask position with numpy : 0.0007207393646240234 nb_pixel_total : 68059 time to create 1 rle with old method : 0.07421374320983887 length of segment : 327 Processing 1 images image shape: (400, 400, 3) min: 12.00000 max: 208.00000 molded_images shape: (1, 640, 640, 3) min: -83.27813 max: 75.17266 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0017294883728027344 nb_pixel_total : 154028 time to create 1 rle with new method : 0.0025031566619873047 length of segment : 401 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -108.21172 max: 150.82656 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 18 time for calcul the mask position with numpy : 4.5299530029296875e-05 nb_pixel_total : 52 time to create 1 rle with old method : 0.00011920928955078125 length of segment : 7 time for calcul the mask position with numpy : 3.314018249511719e-05 nb_pixel_total : 173 time to create 1 rle with old method : 0.0002624988555908203 length of segment : 15 time for calcul the mask position with numpy : 3.123283386230469e-05 nb_pixel_total : 205 time to create 1 rle with old method : 0.00030517578125 length of segment : 14 time for calcul the mask position with numpy : 3.147125244140625e-05 nb_pixel_total : 169 time to create 1 rle with old method : 0.00022673606872558594 length of segment : 25 time for calcul the mask position with numpy : 3.075599670410156e-05 nb_pixel_total : 124 time to create 1 rle with old method : 0.00020885467529296875 length of segment : 15 time for calcul the mask position with numpy : 5.793571472167969e-05 nb_pixel_total : 951 time to create 1 rle with old method : 0.0018002986907958984 length of segment : 39 time for calcul the mask position with numpy : 4.2438507080078125e-05 nb_pixel_total : 368 time to create 1 rle with old method : 0.0005099773406982422 length of segment : 29 time for calcul the mask position with numpy : 2.9802322387695312e-05 nb_pixel_total : 64 time to create 1 rle with old method : 0.00011324882507324219 length of segment : 10 time for calcul the mask position with numpy : 3.075599670410156e-05 nb_pixel_total : 131 time to create 1 rle with old method : 0.00021266937255859375 length of segment : 15 time for calcul the mask position with numpy : 2.9325485229492188e-05 nb_pixel_total : 108 time to create 1 rle with old method : 0.00015163421630859375 length of segment : 22 time for calcul the mask position with numpy : 3.5762786865234375e-05 nb_pixel_total : 512 time to create 1 rle with old method : 0.000732421875 length of segment : 12 time for calcul the mask position with numpy : 3.7670135498046875e-05 nb_pixel_total : 363 time to create 1 rle with old method : 0.0005335807800292969 length of segment : 19 time for calcul the mask position with numpy : 7.653236389160156e-05 nb_pixel_total : 2443 time to create 1 rle with old method : 0.0029625892639160156 length of segment : 40 time for calcul the mask position with numpy : 5.435943603515625e-05 nb_pixel_total : 1178 time to create 1 rle with old method : 0.0015437602996826172 length of segment : 57 time for calcul the mask position with numpy : 3.719329833984375e-05 nb_pixel_total : 392 time to create 1 rle with old method : 0.0005450248718261719 length of segment : 27 time for calcul the mask position with numpy : 3.0517578125e-05 nb_pixel_total : 73 time to create 1 rle with old method : 0.0001652240753173828 length of segment : 10 time for calcul the mask position with numpy : 2.9802322387695312e-05 nb_pixel_total : 94 time to create 1 rle with old method : 0.000179290771484375 length of segment : 9 time for calcul the mask position with numpy : 0.00011801719665527344 nb_pixel_total : 4214 time to create 1 rle with old method : 0.005056858062744141 length of segment : 90 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 17 time for calcul the mask position with numpy : 6.413459777832031e-05 nb_pixel_total : 824 time to create 1 rle with old method : 0.0011134147644042969 length of segment : 25 time for calcul the mask position with numpy : 4.410743713378906e-05 nb_pixel_total : 623 time to create 1 rle with old method : 0.0008401870727539062 length of segment : 29 time for calcul the mask position with numpy : 4.315376281738281e-05 nb_pixel_total : 550 time to create 1 rle with old method : 0.0006890296936035156 length of segment : 36 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 584 time to create 1 rle with old method : 0.0008146762847900391 length of segment : 58 time for calcul the mask position with numpy : 3.886222839355469e-05 nb_pixel_total : 280 time to create 1 rle with old method : 0.0003871917724609375 length of segment : 22 time for calcul the mask position with numpy : 4.8160552978515625e-05 nb_pixel_total : 318 time to create 1 rle with old method : 0.0004892349243164062 length of segment : 40 time for calcul the mask position with numpy : 3.552436828613281e-05 nb_pixel_total : 229 time to create 1 rle with old method : 0.00032806396484375 length of segment : 17 time for calcul the mask position with numpy : 7.748603820800781e-05 nb_pixel_total : 804 time to create 1 rle with old method : 0.0019412040710449219 length of segment : 33 time for calcul the mask position with numpy : 6.747245788574219e-05 nb_pixel_total : 376 time to create 1 rle with old method : 0.0007784366607666016 length of segment : 14 time for calcul the mask position with numpy : 8.797645568847656e-05 nb_pixel_total : 3179 time to create 1 rle with old method : 0.0037679672241210938 length of segment : 90 time for calcul the mask position with numpy : 3.2901763916015625e-05 nb_pixel_total : 114 time to create 1 rle with old method : 0.000194549560546875 length of segment : 12 time for calcul the mask position with numpy : 5.14984130859375e-05 nb_pixel_total : 1032 time to create 1 rle with old method : 0.0013267993927001953 length of segment : 34 time for calcul the mask position with numpy : 5.316734313964844e-05 nb_pixel_total : 382 time to create 1 rle with old method : 0.0006263256072998047 length of segment : 34 time for calcul the mask position with numpy : 7.152557373046875e-05 nb_pixel_total : 1642 time to create 1 rle with old method : 0.0020856857299804688 length of segment : 33 time for calcul the mask position with numpy : 7.987022399902344e-05 nb_pixel_total : 1728 time to create 1 rle with old method : 0.002239704132080078 length of segment : 30 time for calcul the mask position with numpy : 6.628036499023438e-05 nb_pixel_total : 821 time to create 1 rle with old method : 0.0012030601501464844 length of segment : 75 time for calcul the mask position with numpy : 7.414817810058594e-05 nb_pixel_total : 1723 time to create 1 rle with old method : 0.002316713333129883 length of segment : 36 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -115.99297 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 0.0001544952392578125 nb_pixel_total : 4994 time to create 1 rle with old method : 0.00592041015625 length of segment : 162 time for calcul the mask position with numpy : 5.054473876953125e-05 nb_pixel_total : 27 time to create 1 rle with old method : 7.677078247070312e-05 length of segment : 10 time for calcul the mask position with numpy : 0.0003619194030761719 nb_pixel_total : 12136 time to create 1 rle with old method : 0.014365434646606445 length of segment : 322 time for calcul the mask position with numpy : 0.00012087821960449219 nb_pixel_total : 763 time to create 1 rle with old method : 0.0015158653259277344 length of segment : 39 time for calcul the mask position with numpy : 0.0001761913299560547 nb_pixel_total : 1797 time to create 1 rle with old method : 0.0029213428497314453 length of segment : 122 time for calcul the mask position with numpy : 5.14984130859375e-05 nb_pixel_total : 818 time to create 1 rle with old method : 0.0010418891906738281 length of segment : 37 time for calcul the mask position with numpy : 5.1975250244140625e-05 nb_pixel_total : 1270 time to create 1 rle with old method : 0.0015897750854492188 length of segment : 40 time for calcul the mask position with numpy : 5.435943603515625e-05 nb_pixel_total : 1331 time to create 1 rle with old method : 0.0016798973083496094 length of segment : 43 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 149.41641 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 7.82012939453125e-05 nb_pixel_total : 488 time to create 1 rle with old method : 0.0010063648223876953 length of segment : 35 length of segment : 0 time for calcul the mask position with numpy : 0.00022125244140625 nb_pixel_total : 4118 time to create 1 rle with old method : 0.00745081901550293 length of segment : 199 time for calcul the mask position with numpy : 9.298324584960938e-05 nb_pixel_total : 1836 time to create 1 rle with old method : 0.0022575855255126953 length of segment : 86 time for calcul the mask position with numpy : 3.933906555175781e-05 nb_pixel_total : 265 time to create 1 rle with old method : 0.00034546852111816406 length of segment : 32 time for calcul the mask position with numpy : 0.00021910667419433594 nb_pixel_total : 3296 time to create 1 rle with old method : 0.004297494888305664 length of segment : 71 time for calcul the mask position with numpy : 0.0003230571746826172 nb_pixel_total : 1851 time to create 1 rle with old method : 0.0024454593658447266 length of segment : 52 time for calcul the mask position with numpy : 4.267692565917969e-05 nb_pixel_total : 349 time to create 1 rle with old method : 0.0004899501800537109 length of segment : 31 time for calcul the mask position with numpy : 7.939338684082031e-05 nb_pixel_total : 1841 time to create 1 rle with old method : 0.002333402633666992 length of segment : 89 Processing 1 images image shape: (280, 400, 3) min: 7.00000 max: 208.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 79.93203 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 5.9604644775390625e-05 nb_pixel_total : 432 time to create 1 rle with old method : 0.0005934238433837891 length of segment : 49 time for calcul the mask position with numpy : 0.0010497570037841797 nb_pixel_total : 107178 time to create 1 rle with old method : 0.13093328475952148 length of segment : 281 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.12344 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 14 time for calcul the mask position with numpy : 0.00012302398681640625 nb_pixel_total : 5604 time to create 1 rle with old method : 0.0073163509368896484 length of segment : 84 time for calcul the mask position with numpy : 7.009506225585938e-05 nb_pixel_total : 1694 time to create 1 rle with old method : 0.0023157596588134766 length of segment : 41 time for calcul the mask position with numpy : 5.1021575927734375e-05 nb_pixel_total : 612 time to create 1 rle with old method : 0.0008597373962402344 length of segment : 26 time for calcul the mask position with numpy : 6.937980651855469e-05 nb_pixel_total : 1599 time to create 1 rle with old method : 0.002190828323364258 length of segment : 51 time for calcul the mask position with numpy : 4.863739013671875e-05 nb_pixel_total : 255 time to create 1 rle with old method : 0.00040841102600097656 length of segment : 35 time for calcul the mask position with numpy : 3.600120544433594e-05 nb_pixel_total : 190 time to create 1 rle with old method : 0.00033593177795410156 length of segment : 14 time for calcul the mask position with numpy : 4.792213439941406e-05 nb_pixel_total : 785 time to create 1 rle with old method : 0.0010972023010253906 length of segment : 31 time for calcul the mask position with numpy : 0.0001704692840576172 nb_pixel_total : 8154 time to create 1 rle with old method : 0.009922504425048828 length of segment : 160 time for calcul the mask position with numpy : 6.4849853515625e-05 nb_pixel_total : 606 time to create 1 rle with old method : 0.0008509159088134766 length of segment : 33 time for calcul the mask position with numpy : 4.57763671875e-05 nb_pixel_total : 540 time to create 1 rle with old method : 0.0007688999176025391 length of segment : 30 time for calcul the mask position with numpy : 3.528594970703125e-05 nb_pixel_total : 205 time to create 1 rle with old method : 0.0003044605255126953 length of segment : 16 time for calcul the mask position with numpy : 3.1948089599609375e-05 nb_pixel_total : 177 time to create 1 rle with old method : 0.0002713203430175781 length of segment : 19 time for calcul the mask position with numpy : 4.267692565917969e-05 nb_pixel_total : 625 time to create 1 rle with old method : 0.0008411407470703125 length of segment : 28 time for calcul the mask position with numpy : 4.315376281738281e-05 nb_pixel_total : 807 time to create 1 rle with old method : 0.0010731220245361328 length of segment : 32 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 147.24062 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 23 time for calcul the mask position with numpy : 5.7220458984375e-05 nb_pixel_total : 524 time to create 1 rle with old method : 0.0006959438323974609 length of segment : 30 time for calcul the mask position with numpy : 3.528594970703125e-05 nb_pixel_total : 194 time to create 1 rle with old method : 0.00027823448181152344 length of segment : 19 time for calcul the mask position with numpy : 3.790855407714844e-05 nb_pixel_total : 297 time to create 1 rle with old method : 0.00046539306640625 length of segment : 16 time for calcul the mask position with numpy : 3.838539123535156e-05 nb_pixel_total : 336 time to create 1 rle with old method : 0.0005252361297607422 length of segment : 19 time for calcul the mask position with numpy : 4.100799560546875e-05 nb_pixel_total : 619 time to create 1 rle with old method : 0.0008077621459960938 length of segment : 30 time for calcul the mask position with numpy : 3.3855438232421875e-05 nb_pixel_total : 194 time to create 1 rle with old method : 0.0002837181091308594 length of segment : 16 time for calcul the mask position with numpy : 3.218650817871094e-05 nb_pixel_total : 218 time to create 1 rle with old method : 0.00028514862060546875 length of segment : 22 time for calcul the mask position with numpy : 3.0994415283203125e-05 nb_pixel_total : 148 time to create 1 rle with old method : 0.00023245811462402344 length of segment : 16 time for calcul the mask position with numpy : 4.5299530029296875e-05 nb_pixel_total : 366 time to create 1 rle with old method : 0.0007221698760986328 length of segment : 22 time for calcul the mask position with numpy : 4.220008850097656e-05 nb_pixel_total : 125 time to create 1 rle with old method : 0.0003223419189453125 length of segment : 15 time for calcul the mask position with numpy : 3.695487976074219e-05 nb_pixel_total : 72 time to create 1 rle with old method : 0.0002532005310058594 length of segment : 8 time for calcul the mask position with numpy : 3.8623809814453125e-05 nb_pixel_total : 125 time to create 1 rle with old method : 0.00026416778564453125 length of segment : 15 time for calcul the mask position with numpy : 4.029273986816406e-05 nb_pixel_total : 157 time to create 1 rle with old method : 0.00032329559326171875 length of segment : 17 time for calcul the mask position with numpy : 3.886222839355469e-05 nb_pixel_total : 181 time to create 1 rle with old method : 0.0003628730773925781 length of segment : 15 time for calcul the mask position with numpy : 4.9114227294921875e-05 nb_pixel_total : 527 time to create 1 rle with old method : 0.0009489059448242188 length of segment : 34 time for calcul the mask position with numpy : 3.8623809814453125e-05 nb_pixel_total : 96 time to create 1 rle with old method : 0.00024509429931640625 length of segment : 7 time for calcul the mask position with numpy : 6.270408630371094e-05 nb_pixel_total : 462 time to create 1 rle with old method : 0.0010476112365722656 length of segment : 29 time for calcul the mask position with numpy : 5.412101745605469e-05 nb_pixel_total : 175 time to create 1 rle with old method : 0.0005083084106445312 length of segment : 10 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 133 time to create 1 rle with old method : 0.0002880096435546875 length of segment : 14 time for calcul the mask position with numpy : 4.744529724121094e-05 nb_pixel_total : 511 time to create 1 rle with old method : 0.0009489059448242188 length of segment : 23 time for calcul the mask position with numpy : 4.00543212890625e-05 nb_pixel_total : 187 time to create 1 rle with old method : 0.00043201446533203125 length of segment : 14 time for calcul the mask position with numpy : 4.57763671875e-05 nb_pixel_total : 515 time to create 1 rle with old method : 0.0009238719940185547 length of segment : 33 time for calcul the mask position with numpy : 4.0531158447265625e-05 nb_pixel_total : 195 time to create 1 rle with old method : 0.00039577484130859375 length of segment : 13 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.52969 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 14 time for calcul the mask position with numpy : 4.839897155761719e-05 nb_pixel_total : 243 time to create 1 rle with old method : 0.0003483295440673828 length of segment : 20 time for calcul the mask position with numpy : 5.91278076171875e-05 nb_pixel_total : 1600 time to create 1 rle with old method : 0.0019388198852539062 length of segment : 40 time for calcul the mask position with numpy : 4.9591064453125e-05 nb_pixel_total : 312 time to create 1 rle with old method : 0.00045037269592285156 length of segment : 23 time for calcul the mask position with numpy : 5.650520324707031e-05 nb_pixel_total : 537 time to create 1 rle with old method : 0.0006830692291259766 length of segment : 69 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 257 time to create 1 rle with old method : 0.0003688335418701172 length of segment : 20 time for calcul the mask position with numpy : 4.0531158447265625e-05 nb_pixel_total : 240 time to create 1 rle with old method : 0.0003390312194824219 length of segment : 27 time for calcul the mask position with numpy : 3.2901763916015625e-05 nb_pixel_total : 99 time to create 1 rle with old method : 0.00017881393432617188 length of segment : 10 time for calcul the mask position with numpy : 3.695487976074219e-05 nb_pixel_total : 431 time to create 1 rle with old method : 0.0005903244018554688 length of segment : 19 time for calcul the mask position with numpy : 4.982948303222656e-05 nb_pixel_total : 599 time to create 1 rle with old method : 0.0009150505065917969 length of segment : 34 time for calcul the mask position with numpy : 8.130073547363281e-05 nb_pixel_total : 515 time to create 1 rle with old method : 0.0007376670837402344 length of segment : 38 time for calcul the mask position with numpy : 3.790855407714844e-05 nb_pixel_total : 132 time to create 1 rle with old method : 0.000217437744140625 length of segment : 15 length of segment : 0 time for calcul the mask position with numpy : 4.744529724121094e-05 nb_pixel_total : 429 time to create 1 rle with old method : 0.0006577968597412109 length of segment : 70 time for calcul the mask position with numpy : 8.058547973632812e-05 nb_pixel_total : 1599 time to create 1 rle with old method : 0.0031032562255859375 length of segment : 76 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 135.85000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 3.981590270996094e-05 nb_pixel_total : 53 time to create 1 rle with old method : 0.00011086463928222656 length of segment : 10 time for calcul the mask position with numpy : 7.128715515136719e-05 nb_pixel_total : 985 time to create 1 rle with old method : 0.0014278888702392578 length of segment : 89 time for calcul the mask position with numpy : 8.392333984375e-05 nb_pixel_total : 479 time to create 1 rle with old method : 0.001132965087890625 length of segment : 26 time for calcul the mask position with numpy : 3.647804260253906e-05 nb_pixel_total : 44 time to create 1 rle with old method : 0.00010752677917480469 length of segment : 7 Detection mask done ! Trying to reset tf kernel 754038 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1180 tf kernel not reseted sub process len(results) : 260 len(list_Values) 260 None max_time_sub_proc : 3600 parent process len(results) : 0 len(list_Values) 260 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 6469 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Catched exception ! Connect or reconnect ! Number saved : None batch 1 Loaded 260 chid ids of type : 4228 Number RLEs to save : 12713 save missing photos in datou_result : time spend for datou_step_exec : 25.973644733428955 time spend to save output : 1.0429422855377197 total time spend for step 1 : 27.016587018966675 step2:brightness Fri Feb 7 00:20:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1738884029_753391_1335305597_764ce8a39d953053c27c9e4ca6e25e74.jpg treat image : temp/1738884029_753391_1335305578_7ebdcad40d72eb046bd629643012c659.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 2 time used for this insertion : 0.013000965118408203 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 2 time used for this insertion : 0.012545108795166016 save missing photos in datou_result : time spend for datou_step_exec : 0.5687482357025146 time spend to save output : 0.03016972541809082 total time spend for step 2 : 0.5989179611206055 step3:blur_detection Fri Feb 7 00:20:57 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1738884029_753391_1335305597_764ce8a39d953053c27c9e4ca6e25e74.jpg resize: (1080, 1920) 1335305597 -7.787097480236104 treat image : temp/1738884029_753391_1335305578_7ebdcad40d72eb046bd629643012c659.jpg resize: (1080, 1920) 1335305578 -7.767715566346525 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 2 time used for this insertion : 0.011906862258911133 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 2 time used for this insertion : 0.011946678161621094 save missing photos in datou_result : time spend for datou_step_exec : 1.898374319076538 time spend to save output : 0.029034852981567383 total time spend for step 3 : 1.9274091720581055 step4:rle_unique_nms_with_priority Fri Feb 7 00:20:59 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 260 chid ids of type : 4228 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 132 nb_hashtags : 7 time to prepare the origin masks : 1.1223108768463135 time for calcul the mask position with numpy : 0.08342361450195312 nb_pixel_total : 1772961 time to create 1 rle with new method : 0.5204522609710693 time for calcul the mask position with numpy : 0.008120059967041016 nb_pixel_total : 2501 time to create 1 rle with old method : 0.0027260780334472656 time for calcul the mask position with numpy : 0.00783395767211914 nb_pixel_total : 2701 time to create 1 rle with old method : 0.0029706954956054688 time for calcul the mask position with numpy : 0.007729768753051758 nb_pixel_total : 13915 time to create 1 rle with old method : 0.014178276062011719 time for calcul the mask position with numpy : 0.007969379425048828 nb_pixel_total : 977 time to create 1 rle with old method : 0.0010349750518798828 time for calcul the mask position with numpy : 0.007996797561645508 nb_pixel_total : 263 time to create 1 rle with old method : 0.0003247261047363281 time for calcul the mask position with numpy : 0.008216381072998047 nb_pixel_total : 372 time to create 1 rle with old method : 0.0004105567932128906 time for calcul the mask position with numpy : 0.008205652236938477 nb_pixel_total : 12 time to create 1 rle with old method : 2.6226043701171875e-05 time for calcul the mask position with numpy : 0.008114099502563477 nb_pixel_total : 55262 time to create 1 rle with old method : 0.06376218795776367 time for calcul the mask position with numpy : 0.008447647094726562 nb_pixel_total : 2461 time to create 1 rle with old method : 0.0027959346771240234 time for calcul the mask position with numpy : 0.008299827575683594 nb_pixel_total : 101 time to create 1 rle with old method : 0.00013709068298339844 time for calcul the mask position with numpy : 0.008202552795410156 nb_pixel_total : 13 time to create 1 rle with old method : 3.147125244140625e-05 time for calcul the mask position with numpy : 0.008019685745239258 nb_pixel_total : 10719 time to create 1 rle with old method : 0.01114344596862793 time for calcul the mask position with numpy : 0.008060216903686523 nb_pixel_total : 1139 time to create 1 rle with old method : 0.001291036605834961 time for calcul the mask position with numpy : 0.008014440536499023 nb_pixel_total : 3276 time to create 1 rle with old method : 0.00347900390625 time for calcul the mask position with numpy : 0.007633209228515625 nb_pixel_total : 1386 time to create 1 rle with old method : 0.0015163421630859375 time for calcul the mask position with numpy : 0.007857561111450195 nb_pixel_total : 189 time to create 1 rle with old method : 0.0001888275146484375 time for calcul the mask position with numpy : 0.0078582763671875 nb_pixel_total : 415 time to create 1 rle with old method : 0.0004725456237792969 time for calcul the mask position with numpy : 0.007805824279785156 nb_pixel_total : 97 time to create 1 rle with old method : 0.00011110305786132812 time for calcul the mask position with numpy : 0.007744789123535156 nb_pixel_total : 486 time to create 1 rle with old method : 0.0005724430084228516 time for calcul the mask position with numpy : 0.007792949676513672 nb_pixel_total : 1600 time to create 1 rle with old method : 0.001651763916015625 time for calcul the mask position with numpy : 0.00758671760559082 nb_pixel_total : 1442 time to create 1 rle with old method : 0.0015573501586914062 time for calcul the mask position with numpy : 0.0076138973236083984 nb_pixel_total : 1611 time to create 1 rle with old method : 0.0017023086547851562 time for calcul the mask position with numpy : 0.0076770782470703125 nb_pixel_total : 9554 time to create 1 rle with old method : 0.00982522964477539 time for calcul the mask position with numpy : 0.007920980453491211 nb_pixel_total : 1328 time to create 1 rle with old method : 0.0014145374298095703 time for calcul the mask position with numpy : 0.007745265960693359 nb_pixel_total : 490 time to create 1 rle with old method : 0.000518798828125 time for calcul the mask position with numpy : 0.007912635803222656 nb_pixel_total : 1071 time to create 1 rle with old method : 0.0012879371643066406 time for calcul the mask position with numpy : 0.00810861587524414 nb_pixel_total : 106 time to create 1 rle with old method : 0.00012636184692382812 time for calcul the mask position with numpy : 0.008045673370361328 nb_pixel_total : 322 time to create 1 rle with old method : 0.00033473968505859375 time for calcul the mask position with numpy : 0.007903575897216797 nb_pixel_total : 707 time to create 1 rle with old method : 0.0007243156433105469 time for calcul the mask position with numpy : 0.008953571319580078 nb_pixel_total : 1677 time to create 1 rle with old method : 0.0019626617431640625 time for calcul the mask position with numpy : 0.007884502410888672 nb_pixel_total : 38 time to create 1 rle with old method : 9.417533874511719e-05 time for calcul the mask position with numpy : 0.007965564727783203 nb_pixel_total : 213 time to create 1 rle with old method : 0.00023651123046875 time for calcul the mask position with numpy : 0.007929086685180664 nb_pixel_total : 27 time to create 1 rle with old method : 5.316734313964844e-05 time for calcul the mask position with numpy : 0.008159637451171875 nb_pixel_total : 10538 time to create 1 rle with old method : 0.012364864349365234 time for calcul the mask position with numpy : 0.008074045181274414 nb_pixel_total : 867 time to create 1 rle with old method : 0.0009911060333251953 time for calcul the mask position with numpy : 0.008146047592163086 nb_pixel_total : 2467 time to create 1 rle with old method : 0.0026884078979492188 time for calcul the mask position with numpy : 0.008006811141967773 nb_pixel_total : 598 time to create 1 rle with old method : 0.0006411075592041016 time for calcul the mask position with numpy : 0.0078125 nb_pixel_total : 304 time to create 1 rle with old method : 0.00042176246643066406 time for calcul the mask position with numpy : 0.007654905319213867 nb_pixel_total : 1380 time to create 1 rle with old method : 0.0014309883117675781 time for calcul the mask position with numpy : 0.007672786712646484 nb_pixel_total : 15 time to create 1 rle with old method : 5.2928924560546875e-05 time for calcul the mask position with numpy : 0.007738351821899414 nb_pixel_total : 199 time to create 1 rle with old method : 0.0002167224884033203 time for calcul the mask position with numpy : 0.007897615432739258 nb_pixel_total : 855 time to create 1 rle with old method : 0.0009241104125976562 time for calcul the mask position with numpy : 0.007653713226318359 nb_pixel_total : 125 time to create 1 rle with old method : 0.000141143798828125 time for calcul the mask position with numpy : 0.008811712265014648 nb_pixel_total : 1024 time to create 1 rle with old method : 0.0011136531829833984 time for calcul the mask position with numpy : 0.007909536361694336 nb_pixel_total : 6345 time to create 1 rle with old method : 0.006953001022338867 time for calcul the mask position with numpy : 0.0077707767486572266 nb_pixel_total : 242 time to create 1 rle with old method : 0.0002715587615966797 time for calcul the mask position with numpy : 0.008034467697143555 nb_pixel_total : 251 time to create 1 rle with old method : 0.0002830028533935547 time for calcul the mask position with numpy : 0.007929325103759766 nb_pixel_total : 2184 time to create 1 rle with old method : 0.002356290817260742 time for calcul the mask position with numpy : 0.007835149765014648 nb_pixel_total : 3502 time to create 1 rle with old method : 0.0037577152252197266 time for calcul the mask position with numpy : 0.007954835891723633 nb_pixel_total : 355 time to create 1 rle with old method : 0.0004096031188964844 time for calcul the mask position with numpy : 0.007976055145263672 nb_pixel_total : 375 time to create 1 rle with old method : 0.0004742145538330078 time for calcul the mask position with numpy : 0.007735013961791992 nb_pixel_total : 1056 time to create 1 rle with old method : 0.0010743141174316406 time for calcul the mask position with numpy : 0.00775909423828125 nb_pixel_total : 1046 time to create 1 rle with old method : 0.001125335693359375 time for calcul the mask position with numpy : 0.007746219635009766 nb_pixel_total : 328 time to create 1 rle with old method : 0.0003478527069091797 time for calcul the mask position with numpy : 0.007910490036010742 nb_pixel_total : 666 time to create 1 rle with old method : 0.0007216930389404297 time for calcul the mask position with numpy : 0.00775146484375 nb_pixel_total : 20 time to create 1 rle with old method : 2.956390380859375e-05 time for calcul the mask position with numpy : 0.0077877044677734375 nb_pixel_total : 470 time to create 1 rle with old method : 0.0005540847778320312 time for calcul the mask position with numpy : 0.0077283382415771484 nb_pixel_total : 872 time to create 1 rle with old method : 0.0009927749633789062 time for calcul the mask position with numpy : 0.00788116455078125 nb_pixel_total : 376 time to create 1 rle with old method : 0.0003979206085205078 time for calcul the mask position with numpy : 0.007980823516845703 nb_pixel_total : 30 time to create 1 rle with old method : 5.316734313964844e-05 time for calcul the mask position with numpy : 0.00791621208190918 nb_pixel_total : 4099 time to create 1 rle with old method : 0.004327297210693359 time for calcul the mask position with numpy : 0.00794363021850586 nb_pixel_total : 175 time to create 1 rle with old method : 0.00019478797912597656 time for calcul the mask position with numpy : 0.00816202163696289 nb_pixel_total : 1356 time to create 1 rle with old method : 0.0015723705291748047 time for calcul the mask position with numpy : 0.007729053497314453 nb_pixel_total : 80 time to create 1 rle with old method : 0.0001227855682373047 time for calcul the mask position with numpy : 0.00767970085144043 nb_pixel_total : 1740 time to create 1 rle with old method : 0.0019326210021972656 time for calcul the mask position with numpy : 0.007756471633911133 nb_pixel_total : 1659 time to create 1 rle with old method : 0.0017542839050292969 time for calcul the mask position with numpy : 0.008067607879638672 nb_pixel_total : 86 time to create 1 rle with old method : 0.00019693374633789062 time for calcul the mask position with numpy : 0.009580850601196289 nb_pixel_total : 118 time to create 1 rle with old method : 0.00015211105346679688 time for calcul the mask position with numpy : 0.008014202117919922 nb_pixel_total : 259 time to create 1 rle with old method : 0.00030612945556640625 time for calcul the mask position with numpy : 0.008118152618408203 nb_pixel_total : 66 time to create 1 rle with old method : 7.939338684082031e-05 time for calcul the mask position with numpy : 0.00832366943359375 nb_pixel_total : 363 time to create 1 rle with old method : 0.00040602684020996094 time for calcul the mask position with numpy : 0.008209943771362305 nb_pixel_total : 21 time to create 1 rle with old method : 3.4332275390625e-05 time for calcul the mask position with numpy : 0.01367950439453125 nb_pixel_total : 54 time to create 1 rle with old method : 0.000125885009765625 time for calcul the mask position with numpy : 0.01639270782470703 nb_pixel_total : 501 time to create 1 rle with old method : 0.0017323493957519531 time for calcul the mask position with numpy : 0.014902114868164062 nb_pixel_total : 106084 time to create 1 rle with old method : 0.11850929260253906 time for calcul the mask position with numpy : 0.008851289749145508 nb_pixel_total : 1734 time to create 1 rle with old method : 0.0020203590393066406 time for calcul the mask position with numpy : 0.00864553451538086 nb_pixel_total : 460 time to create 1 rle with old method : 0.0005772113800048828 time for calcul the mask position with numpy : 0.00855112075805664 nb_pixel_total : 361 time to create 1 rle with old method : 0.0004515647888183594 time for calcul the mask position with numpy : 0.00859379768371582 nb_pixel_total : 2 time to create 1 rle with old method : 4.100799560546875e-05 time for calcul the mask position with numpy : 0.008503913879394531 nb_pixel_total : 105 time to create 1 rle with old method : 0.0001468658447265625 time for calcul the mask position with numpy : 0.008757829666137695 nb_pixel_total : 1497 time to create 1 rle with old method : 0.0017728805541992188 time for calcul the mask position with numpy : 0.008533000946044922 nb_pixel_total : 237 time to create 1 rle with old method : 0.00032138824462890625 time for calcul the mask position with numpy : 0.008489608764648438 nb_pixel_total : 1547 time to create 1 rle with old method : 0.0018117427825927734 time for calcul the mask position with numpy : 0.00842142105102539 nb_pixel_total : 295 time to create 1 rle with old method : 0.0005288124084472656 time for calcul the mask position with numpy : 0.010436534881591797 nb_pixel_total : 407 time to create 1 rle with old method : 0.0006802082061767578 time for calcul the mask position with numpy : 0.010965347290039062 nb_pixel_total : 611 time to create 1 rle with old method : 0.0010046958923339844 time for calcul the mask position with numpy : 0.010061979293823242 nb_pixel_total : 131 time to create 1 rle with old method : 0.0002410411834716797 time for calcul the mask position with numpy : 0.008336305618286133 nb_pixel_total : 161 time to create 1 rle with old method : 0.0001888275146484375 time for calcul the mask position with numpy : 0.008352279663085938 nb_pixel_total : 189 time to create 1 rle with old method : 0.0002644062042236328 time for calcul the mask position with numpy : 0.008417844772338867 nb_pixel_total : 282 time to create 1 rle with old method : 0.00034046173095703125 time for calcul the mask position with numpy : 0.008215665817260742 nb_pixel_total : 226 time to create 1 rle with old method : 0.0002789497375488281 time for calcul the mask position with numpy : 0.008423089981079102 nb_pixel_total : 295 time to create 1 rle with old method : 0.0003459453582763672 time for calcul the mask position with numpy : 0.008566856384277344 nb_pixel_total : 170 time to create 1 rle with old method : 0.0002205371856689453 time for calcul the mask position with numpy : 0.008513927459716797 nb_pixel_total : 253 time to create 1 rle with old method : 0.0003573894500732422 time for calcul the mask position with numpy : 0.00853276252746582 nb_pixel_total : 201 time to create 1 rle with old method : 0.00023794174194335938 time for calcul the mask position with numpy : 0.008642911911010742 nb_pixel_total : 415 time to create 1 rle with old method : 0.0005166530609130859 time for calcul the mask position with numpy : 0.00849771499633789 nb_pixel_total : 767 time to create 1 rle with old method : 0.0009150505065917969 time for calcul the mask position with numpy : 0.008453607559204102 nb_pixel_total : 678 time to create 1 rle with old method : 0.0008397102355957031 time for calcul the mask position with numpy : 0.008438587188720703 nb_pixel_total : 433 time to create 1 rle with old method : 0.0005238056182861328 time for calcul the mask position with numpy : 0.008498191833496094 nb_pixel_total : 187 time to create 1 rle with old method : 0.00024199485778808594 time for calcul the mask position with numpy : 0.008509397506713867 nb_pixel_total : 105 time to create 1 rle with old method : 0.0001468658447265625 time for calcul the mask position with numpy : 0.00854349136352539 nb_pixel_total : 249 time to create 1 rle with old method : 0.0003533363342285156 time for calcul the mask position with numpy : 0.009760618209838867 nb_pixel_total : 243 time to create 1 rle with old method : 0.0002982616424560547 time for calcul the mask position with numpy : 0.008397340774536133 nb_pixel_total : 419 time to create 1 rle with old method : 0.0005042552947998047 time for calcul the mask position with numpy : 0.008490562438964844 nb_pixel_total : 180 time to create 1 rle with old method : 0.0002529621124267578 time for calcul the mask position with numpy : 0.008713960647583008 nb_pixel_total : 1054 time to create 1 rle with old method : 0.0012500286102294922 time for calcul the mask position with numpy : 0.008610725402832031 nb_pixel_total : 77 time to create 1 rle with old method : 0.00012373924255371094 time for calcul the mask position with numpy : 0.008551836013793945 nb_pixel_total : 130 time to create 1 rle with old method : 0.00018358230590820312 time for calcul the mask position with numpy : 0.008556127548217773 nb_pixel_total : 392 time to create 1 rle with old method : 0.000637054443359375 time for calcul the mask position with numpy : 0.008655548095703125 nb_pixel_total : 139 time to create 1 rle with old method : 0.00019407272338867188 time for calcul the mask position with numpy : 0.008536100387573242 nb_pixel_total : 265 time to create 1 rle with old method : 0.00034880638122558594 time for calcul the mask position with numpy : 0.008471965789794922 nb_pixel_total : 226 time to create 1 rle with old method : 0.000278472900390625 time for calcul the mask position with numpy : 0.008573770523071289 nb_pixel_total : 55 time to create 1 rle with old method : 0.00016355514526367188 time for calcul the mask position with numpy : 0.008584260940551758 nb_pixel_total : 1058 time to create 1 rle with old method : 0.0013091564178466797 time for calcul the mask position with numpy : 0.00855708122253418 nb_pixel_total : 317 time to create 1 rle with old method : 0.00041413307189941406 time for calcul the mask position with numpy : 0.008449792861938477 nb_pixel_total : 7687 time to create 1 rle with old method : 0.008557558059692383 time for calcul the mask position with numpy : 0.00834965705871582 nb_pixel_total : 1055 time to create 1 rle with old method : 0.0012326240539550781 time for calcul the mask position with numpy : 0.008091449737548828 nb_pixel_total : 111 time to create 1 rle with old method : 0.00013875961303710938 time for calcul the mask position with numpy : 0.008862733840942383 nb_pixel_total : 476 time to create 1 rle with old method : 0.0008587837219238281 time for calcul the mask position with numpy : 0.008851766586303711 nb_pixel_total : 52 time to create 1 rle with old method : 0.0001246929168701172 time for calcul the mask position with numpy : 0.008317947387695312 nb_pixel_total : 758 time to create 1 rle with old method : 0.0008914470672607422 time for calcul the mask position with numpy : 0.008124589920043945 nb_pixel_total : 527 time to create 1 rle with old method : 0.0005950927734375 time for calcul the mask position with numpy : 0.008327007293701172 nb_pixel_total : 170 time to create 1 rle with old method : 0.00021719932556152344 time for calcul the mask position with numpy : 0.00811624526977539 nb_pixel_total : 365 time to create 1 rle with old method : 0.0004322528839111328 time for calcul the mask position with numpy : 0.00814199447631836 nb_pixel_total : 505 time to create 1 rle with old method : 0.0006110668182373047 time for calcul the mask position with numpy : 0.008388757705688477 nb_pixel_total : 546 time to create 1 rle with old method : 0.0006666183471679688 time for calcul the mask position with numpy : 0.008318424224853516 nb_pixel_total : 1533 time to create 1 rle with old method : 0.0016372203826904297 time for calcul the mask position with numpy : 0.008433818817138672 nb_pixel_total : 61 time to create 1 rle with old method : 0.000171661376953125 time for calcul the mask position with numpy : 0.008314371109008789 nb_pixel_total : 50 time to create 1 rle with old method : 8.940696716308594e-05 time for calcul the mask position with numpy : 0.008513212203979492 nb_pixel_total : 257 time to create 1 rle with old method : 0.0003349781036376953 time for calcul the mask position with numpy : 0.008321285247802734 nb_pixel_total : 201 time to create 1 rle with old method : 0.0002529621124267578 time for calcul the mask position with numpy : 0.008344888687133789 nb_pixel_total : 110 time to create 1 rle with old method : 0.00014209747314453125 create new chi : 2.064854621887207 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.007214784622192383 batch 1 Loaded 133 chid ids of type : 4230 Number RLEs to save : 12702 TO DO : save crop sub photo not yet done ! save time : 0.8311913013458252 nb_obj : 126 nb_hashtags : 8 time to prepare the origin masks : 2.5104496479034424 time for calcul the mask position with numpy : 0.03017401695251465 nb_pixel_total : 1629751 time to create 1 rle with new method : 0.05624270439147949 time for calcul the mask position with numpy : 0.007359504699707031 nb_pixel_total : 438 time to create 1 rle with old method : 0.0005619525909423828 time for calcul the mask position with numpy : 0.006575345993041992 nb_pixel_total : 14469 time to create 1 rle with old method : 0.01624774932861328 time for calcul the mask position with numpy : 0.006724834442138672 nb_pixel_total : 893 time to create 1 rle with old method : 0.001012563705444336 time for calcul the mask position with numpy : 0.006699800491333008 nb_pixel_total : 41 time to create 1 rle with old method : 0.00010156631469726562 time for calcul the mask position with numpy : 0.007579326629638672 nb_pixel_total : 333 time to create 1 rle with old method : 0.00040411949157714844 time for calcul the mask position with numpy : 0.007330656051635742 nb_pixel_total : 54468 time to create 1 rle with old method : 0.05968356132507324 time for calcul the mask position with numpy : 0.006712913513183594 nb_pixel_total : 2763 time to create 1 rle with old method : 0.003194093704223633 time for calcul the mask position with numpy : 0.0071773529052734375 nb_pixel_total : 62 time to create 1 rle with old method : 0.0001068115234375 time for calcul the mask position with numpy : 0.006922721862792969 nb_pixel_total : 2 time to create 1 rle with old method : 2.765655517578125e-05 time for calcul the mask position with numpy : 0.006949663162231445 nb_pixel_total : 41 time to create 1 rle with old method : 7.152557373046875e-05 time for calcul the mask position with numpy : 0.0067958831787109375 nb_pixel_total : 1710 time to create 1 rle with old method : 0.001926422119140625 time for calcul the mask position with numpy : 0.006253719329833984 nb_pixel_total : 211 time to create 1 rle with old method : 0.0002453327178955078 time for calcul the mask position with numpy : 0.006089448928833008 nb_pixel_total : 96 time to create 1 rle with old method : 0.00013256072998046875 time for calcul the mask position with numpy : 0.006116390228271484 nb_pixel_total : 1267 time to create 1 rle with old method : 0.0015349388122558594 time for calcul the mask position with numpy : 0.008520126342773438 nb_pixel_total : 5089 time to create 1 rle with old method : 0.0057070255279541016 time for calcul the mask position with numpy : 0.005918979644775391 nb_pixel_total : 1041 time to create 1 rle with old method : 0.0012645721435546875 time for calcul the mask position with numpy : 0.007443904876708984 nb_pixel_total : 11292 time to create 1 rle with old method : 0.012980937957763672 time for calcul the mask position with numpy : 0.006335020065307617 nb_pixel_total : 472 time to create 1 rle with old method : 0.0005481243133544922 time for calcul the mask position with numpy : 0.007962226867675781 nb_pixel_total : 154028 time to create 1 rle with new method : 0.028832674026489258 time for calcul the mask position with numpy : 0.006122589111328125 nb_pixel_total : 107 time to create 1 rle with old method : 0.00015974044799804688 time for calcul the mask position with numpy : 0.0066645145416259766 nb_pixel_total : 584 time to create 1 rle with old method : 0.0007114410400390625 time for calcul the mask position with numpy : 0.006331443786621094 nb_pixel_total : 39 time to create 1 rle with old method : 0.00011181831359863281 time for calcul the mask position with numpy : 0.005912065505981445 nb_pixel_total : 1642 time to create 1 rle with old method : 0.0017533302307128906 time for calcul the mask position with numpy : 0.005819797515869141 nb_pixel_total : 132 time to create 1 rle with old method : 0.00022935867309570312 time for calcul the mask position with numpy : 0.0059490203857421875 nb_pixel_total : 550 time to create 1 rle with old method : 0.0006346702575683594 time for calcul the mask position with numpy : 0.006282806396484375 nb_pixel_total : 12023 time to create 1 rle with old method : 0.013148069381713867 time for calcul the mask position with numpy : 0.0071868896484375 nb_pixel_total : 818 time to create 1 rle with old method : 0.0009479522705078125 time for calcul the mask position with numpy : 0.006119251251220703 nb_pixel_total : 824 time to create 1 rle with old method : 0.0009756088256835938 time for calcul the mask position with numpy : 0.0060634613037109375 nb_pixel_total : 1270 time to create 1 rle with old method : 0.0014159679412841797 time for calcul the mask position with numpy : 0.007170677185058594 nb_pixel_total : 84 time to create 1 rle with old method : 0.0001232624053955078 time for calcul the mask position with numpy : 0.006223917007446289 nb_pixel_total : 382 time to create 1 rle with old method : 0.0005142688751220703 time for calcul the mask position with numpy : 0.007349729537963867 nb_pixel_total : 205 time to create 1 rle with old method : 0.00024509429931640625 time for calcul the mask position with numpy : 0.010253667831420898 nb_pixel_total : 1059 time to create 1 rle with old method : 0.0012650489807128906 time for calcul the mask position with numpy : 0.011552810668945312 nb_pixel_total : 26 time to create 1 rle with old method : 5.5789947509765625e-05 time for calcul the mask position with numpy : 0.009959697723388672 nb_pixel_total : 131 time to create 1 rle with old method : 0.00020456314086914062 time for calcul the mask position with numpy : 0.01020193099975586 nb_pixel_total : 280 time to create 1 rle with old method : 0.0003364086151123047 time for calcul the mask position with numpy : 0.01079559326171875 nb_pixel_total : 4038 time to create 1 rle with old method : 0.004624128341674805 time for calcul the mask position with numpy : 0.010551214218139648 nb_pixel_total : 4994 time to create 1 rle with old method : 0.0058786869049072266 time for calcul the mask position with numpy : 0.010004043579101562 nb_pixel_total : 366 time to create 1 rle with old method : 0.0004565715789794922 time for calcul the mask position with numpy : 0.009853601455688477 nb_pixel_total : 229 time to create 1 rle with old method : 0.0002758502960205078 time for calcul the mask position with numpy : 0.009875297546386719 nb_pixel_total : 3296 time to create 1 rle with old method : 0.003712892532348633 time for calcul the mask position with numpy : 0.012145757675170898 nb_pixel_total : 804 time to create 1 rle with old method : 0.0009377002716064453 time for calcul the mask position with numpy : 0.009934186935424805 nb_pixel_total : 4118 time to create 1 rle with old method : 0.004729747772216797 time for calcul the mask position with numpy : 0.010334253311157227 nb_pixel_total : 349 time to create 1 rle with old method : 0.000423431396484375 time for calcul the mask position with numpy : 0.010219573974609375 nb_pixel_total : 951 time to create 1 rle with old method : 0.0010943412780761719 time for calcul the mask position with numpy : 0.010198831558227539 nb_pixel_total : 1032 time to create 1 rle with old method : 0.0012013912200927734 time for calcul the mask position with numpy : 0.009750604629516602 nb_pixel_total : 363 time to create 1 rle with old method : 0.0004572868347167969 time for calcul the mask position with numpy : 0.01045083999633789 nb_pixel_total : 318 time to create 1 rle with old method : 0.000396728515625 time for calcul the mask position with numpy : 0.010128021240234375 nb_pixel_total : 821 time to create 1 rle with old method : 0.0010116100311279297 time for calcul the mask position with numpy : 0.010027170181274414 nb_pixel_total : 623 time to create 1 rle with old method : 0.0007388591766357422 time for calcul the mask position with numpy : 0.009995222091674805 nb_pixel_total : 27 time to create 1 rle with old method : 5.507469177246094e-05 time for calcul the mask position with numpy : 0.010195255279541016 nb_pixel_total : 460 time to create 1 rle with old method : 0.0005967617034912109 time for calcul the mask position with numpy : 0.010293006896972656 nb_pixel_total : 488 time to create 1 rle with old method : 0.0005662441253662109 time for calcul the mask position with numpy : 0.010215044021606445 nb_pixel_total : 3179 time to create 1 rle with old method : 0.003667593002319336 time for calcul the mask position with numpy : 0.01258540153503418 nb_pixel_total : 763 time to create 1 rle with old method : 0.0009732246398925781 time for calcul the mask position with numpy : 0.009814977645874023 nb_pixel_total : 265 time to create 1 rle with old method : 0.00028824806213378906 time for calcul the mask position with numpy : 0.009908676147460938 nb_pixel_total : 173 time to create 1 rle with old method : 0.0002148151397705078 time for calcul the mask position with numpy : 0.011389970779418945 nb_pixel_total : 39 time to create 1 rle with old method : 0.00010967254638671875 time for calcul the mask position with numpy : 0.010895729064941406 nb_pixel_total : 1836 time to create 1 rle with old method : 0.002112150192260742 time for calcul the mask position with numpy : 0.011281967163085938 nb_pixel_total : 64 time to create 1 rle with old method : 0.00010275840759277344 time for calcul the mask position with numpy : 0.01121377944946289 nb_pixel_total : 84 time to create 1 rle with old method : 0.0001304149627685547 time for calcul the mask position with numpy : 0.010878801345825195 nb_pixel_total : 73 time to create 1 rle with old method : 0.00012302398681640625 time for calcul the mask position with numpy : 0.010973691940307617 nb_pixel_total : 295 time to create 1 rle with old method : 0.000362396240234375 time for calcul the mask position with numpy : 0.007332801818847656 nb_pixel_total : 108 time to create 1 rle with old method : 0.0001430511474609375 time for calcul the mask position with numpy : 0.006956815719604492 nb_pixel_total : 2443 time to create 1 rle with old method : 0.0025975704193115234 time for calcul the mask position with numpy : 0.006304740905761719 nb_pixel_total : 95 time to create 1 rle with old method : 0.0001423358917236328 time for calcul the mask position with numpy : 0.006451845169067383 nb_pixel_total : 52 time to create 1 rle with old method : 7.796287536621094e-05 time for calcul the mask position with numpy : 0.005901813507080078 nb_pixel_total : 382 time to create 1 rle with old method : 0.000484466552734375 time for calcul the mask position with numpy : 0.006178617477416992 nb_pixel_total : 169 time to create 1 rle with old method : 0.0001933574676513672 time for calcul the mask position with numpy : 0.00590205192565918 nb_pixel_total : 512 time to create 1 rle with old method : 0.000545501708984375 time for calcul the mask position with numpy : 0.005789995193481445 nb_pixel_total : 606 time to create 1 rle with old method : 0.0006492137908935547 time for calcul the mask position with numpy : 0.006248950958251953 nb_pixel_total : 106746 time to create 1 rle with old method : 0.13420987129211426 time for calcul the mask position with numpy : 0.00831460952758789 nb_pixel_total : 1694 time to create 1 rle with old method : 0.003159761428833008 time for calcul the mask position with numpy : 0.007832050323486328 nb_pixel_total : 8 time to create 1 rle with old method : 0.00010085105895996094 time for calcul the mask position with numpy : 0.008214235305786133 nb_pixel_total : 1599 time to create 1 rle with old method : 0.0032892227172851562 time for calcul the mask position with numpy : 0.007889270782470703 nb_pixel_total : 985 time to create 1 rle with old method : 0.001967191696166992 time for calcul the mask position with numpy : 0.007802009582519531 nb_pixel_total : 1599 time to create 1 rle with old method : 0.0030999183654785156 time for calcul the mask position with numpy : 0.007779836654663086 nb_pixel_total : 255 time to create 1 rle with old method : 0.0005850791931152344 time for calcul the mask position with numpy : 0.007876396179199219 nb_pixel_total : 4 time to create 1 rle with old method : 4.673004150390625e-05 time for calcul the mask position with numpy : 0.0076694488525390625 nb_pixel_total : 125 time to create 1 rle with old method : 0.00020170211791992188 time for calcul the mask position with numpy : 0.007530689239501953 nb_pixel_total : 8154 time to create 1 rle with old method : 0.01092076301574707 time for calcul the mask position with numpy : 0.0073506832122802734 nb_pixel_total : 297 time to create 1 rle with old method : 0.0004601478576660156 time for calcul the mask position with numpy : 0.007168769836425781 nb_pixel_total : 195 time to create 1 rle with old method : 0.0002942085266113281 time for calcul the mask position with numpy : 0.007153034210205078 nb_pixel_total : 612 time to create 1 rle with old method : 0.0009045600891113281 time for calcul the mask position with numpy : 0.007945060729980469 nb_pixel_total : 20 time to create 1 rle with old method : 9.369850158691406e-05 time for calcul the mask position with numpy : 0.008136272430419922 nb_pixel_total : 175 time to create 1 rle with old method : 0.00024127960205078125 time for calcul the mask position with numpy : 0.007960796356201172 nb_pixel_total : 511 time to create 1 rle with old method : 0.0007138252258300781 time for calcul the mask position with numpy : 0.007828712463378906 nb_pixel_total : 133 time to create 1 rle with old method : 0.00022840499877929688 time for calcul the mask position with numpy : 0.007035017013549805 nb_pixel_total : 157 time to create 1 rle with old method : 0.000194549560546875 time for calcul the mask position with numpy : 0.006298065185546875 nb_pixel_total : 524 time to create 1 rle with old method : 0.0007147789001464844 time for calcul the mask position with numpy : 0.006619691848754883 nb_pixel_total : 312 time to create 1 rle with old method : 0.0004508495330810547 time for calcul the mask position with numpy : 0.006959438323974609 nb_pixel_total : 218 time to create 1 rle with old method : 0.00033783912658691406 time for calcul the mask position with numpy : 0.0074918270111083984 nb_pixel_total : 132 time to create 1 rle with old method : 0.00021839141845703125 time for calcul the mask position with numpy : 0.007226705551147461 nb_pixel_total : 181 time to create 1 rle with old method : 0.00028586387634277344 time for calcul the mask position with numpy : 0.00770258903503418 nb_pixel_total : 187 time to create 1 rle with old method : 0.0002300739288330078 time for calcul the mask position with numpy : 0.0076715946197509766 nb_pixel_total : 537 time to create 1 rle with old method : 0.0008075237274169922 time for calcul the mask position with numpy : 0.00777888298034668 nb_pixel_total : 619 time to create 1 rle with old method : 0.0009472370147705078 time for calcul the mask position with numpy : 0.007900476455688477 nb_pixel_total : 431 time to create 1 rle with old method : 0.0006549358367919922 time for calcul the mask position with numpy : 0.007787466049194336 nb_pixel_total : 15 time to create 1 rle with old method : 6.580352783203125e-05 time for calcul the mask position with numpy : 0.007921695709228516 nb_pixel_total : 190 time to create 1 rle with old method : 0.00033354759216308594 time for calcul the mask position with numpy : 0.00794363021850586 nb_pixel_total : 257 time to create 1 rle with old method : 0.00040411949157714844 time for calcul the mask position with numpy : 0.007869482040405273 nb_pixel_total : 479 time to create 1 rle with old method : 0.0007061958312988281 time for calcul the mask position with numpy : 0.008060932159423828 nb_pixel_total : 462 time to create 1 rle with old method : 0.0006964206695556641 time for calcul the mask position with numpy : 0.008187532424926758 nb_pixel_total : 44 time to create 1 rle with old method : 9.751319885253906e-05 time for calcul the mask position with numpy : 0.008471488952636719 nb_pixel_total : 240 time to create 1 rle with old method : 0.0003521442413330078 time for calcul the mask position with numpy : 0.007780313491821289 nb_pixel_total : 96 time to create 1 rle with old method : 0.0001747608184814453 time for calcul the mask position with numpy : 0.007561683654785156 nb_pixel_total : 99 time to create 1 rle with old method : 0.00014066696166992188 time for calcul the mask position with numpy : 0.007572650909423828 nb_pixel_total : 148 time to create 1 rle with old method : 0.00024008750915527344 time for calcul the mask position with numpy : 0.00757908821105957 nb_pixel_total : 599 time to create 1 rle with old method : 0.0009431838989257812 time for calcul the mask position with numpy : 0.007678985595703125 nb_pixel_total : 336 time to create 1 rle with old method : 0.0004749298095703125 time for calcul the mask position with numpy : 0.007845401763916016 nb_pixel_total : 5604 time to create 1 rle with old method : 0.007535219192504883 time for calcul the mask position with numpy : 0.007775783538818359 nb_pixel_total : 429 time to create 1 rle with old method : 0.0006890296936035156 time for calcul the mask position with numpy : 0.00800633430480957 nb_pixel_total : 515 time to create 1 rle with old method : 0.0008087158203125 time for calcul the mask position with numpy : 0.007757663726806641 nb_pixel_total : 125 time to create 1 rle with old method : 0.000225067138671875 time for calcul the mask position with numpy : 0.007579803466796875 nb_pixel_total : 432 time to create 1 rle with old method : 0.0005998611450195312 time for calcul the mask position with numpy : 0.007505655288696289 nb_pixel_total : 30 time to create 1 rle with old method : 6.4849853515625e-05 time for calcul the mask position with numpy : 0.007913589477539062 nb_pixel_total : 785 time to create 1 rle with old method : 0.0011570453643798828 time for calcul the mask position with numpy : 0.007772922515869141 nb_pixel_total : 194 time to create 1 rle with old method : 0.00028967857360839844 time for calcul the mask position with numpy : 0.0077059268951416016 nb_pixel_total : 366 time to create 1 rle with old method : 0.0005269050598144531 time for calcul the mask position with numpy : 0.0074541568756103516 nb_pixel_total : 1600 time to create 1 rle with old method : 0.0023164749145507812 time for calcul the mask position with numpy : 0.007708549499511719 nb_pixel_total : 18 time to create 1 rle with old method : 6.151199340820312e-05 time for calcul the mask position with numpy : 0.007616996765136719 nb_pixel_total : 527 time to create 1 rle with old method : 0.0007960796356201172 time for calcul the mask position with numpy : 0.00751948356628418 nb_pixel_total : 72 time to create 1 rle with old method : 0.00013446807861328125 time for calcul the mask position with numpy : 0.007535219192504883 nb_pixel_total : 53 time to create 1 rle with old method : 0.00010728836059570312 time for calcul the mask position with numpy : 0.007552146911621094 nb_pixel_total : 243 time to create 1 rle with old method : 0.00045871734619140625 time for calcul the mask position with numpy : 0.007674217224121094 nb_pixel_total : 194 time to create 1 rle with old method : 0.0003108978271484375 create new chi : 1.490408182144165 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.001980304718017578 batch 1 Loaded 127 chid ids of type : 4230 Number RLEs to save : 12096 TO DO : save crop sub photo not yet done ! save time : 0.8295376300811768 map_output_result : {1335305597: (0.0, 'Should be the crop_list due to order', 0.0), 1335305578: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1335305597, 1335305578] Looping around the photos to save general results len do output : 2 /1335305597.Didn't retrieve data . /1335305578.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2560851') ('4234', '20313900', '1335305597', None, None, None, None, None, '2560851') ('4234', None, None, None, None, None, None, None, '2560851') ('4234', '20313900', '1335305578', None, None, None, None, None, '2560851') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.01256108283996582 save_final save missing photos in datou_result : time spend for datou_step_exec : 9.621882915496826 time spend to save output : 0.0128173828125 total time spend for step 4 : 9.634700298309326 step5:crop_condition Fri Feb 7 00:21:08 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4230 Loading chi in step crop for list_pids : 2 ! batch 1 Loaded 260 chid ids of type : 4230 begin to crop the class : papier param for this class : {'min_score': 0.6} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 92 About to insert : list_path_to_insert length 92 new photo from crops ! About to upload 92 photos upload in portfolio : 4869462 init cache_photo without model_param we have 92 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738884070_753391 we have uploaded 92 photos in the portfolio 4869462 time of upload the photos Elapsed time : 19.401835680007935 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.6} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 31 About to insert : list_path_to_insert length 31 new photo from crops ! About to upload 31 photos upload in portfolio : 4869462 init cache_photo without model_param we have 31 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738884090_753391 we have uploaded 31 photos in the portfolio 4869462 time of upload the photos Elapsed time : 7.0543084144592285 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.6} filtre for class : metal hashtag_id of this class : 492628673 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738884097_753391 we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.7703232765197754 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.6} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 16 About to insert : list_path_to_insert length 16 new photo from crops ! About to upload 16 photos upload in portfolio : 4869462 init cache_photo without model_param we have 16 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738884099_753391 we have uploaded 16 photos in the portfolio 4869462 time of upload the photos Elapsed time : 5.077932834625244 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.6} filtre for class : autre hashtag_id of this class : 494826614 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 32 About to insert : list_path_to_insert length 32 new photo from crops ! About to upload 32 photos upload in portfolio : 4869462 init cache_photo without model_param we have 32 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738884105_753391 we have uploaded 32 photos in the portfolio 4869462 time of upload the photos Elapsed time : 12.632435321807861 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.6} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.6} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1335305597, 1335305578] Looping around the photos to save general results len do output : 173 /1335321049Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321050Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321051Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321052Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321053Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321054Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321055Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321056Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321057Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321058Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321059Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321060Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321061Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321062Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321063Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321064Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321065Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321066Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321067Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321068Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321069Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321070Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321071Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321072Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321073Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321074Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321075Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321076Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321077Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321078Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321079Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321080Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321081Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321082Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321083Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321084Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321085Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321086Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321087Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321088Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321089Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321090Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321091Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321092Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321093Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321094Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321095Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321096Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321097Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321098Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321099Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321100Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321101Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321102Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321104Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321105Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321106Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321107Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321108Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321109Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321110Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321111Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321112Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321113Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321114Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321115Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321116Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321117Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321118Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321119Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321120Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321121Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321122Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321123Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321124Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321125Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321126Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321127Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321128Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321129Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321130Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321132Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321133Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321134Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321135Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321136Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321137Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321138Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321139Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321140Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321141Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321142Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321145Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321146Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321147Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321148Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321149Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321150Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321151Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321152Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321153Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321154Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321155Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321156Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321157Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321158Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321159Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321160Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321161Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321162Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321163Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321164Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321165Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321166Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321168Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321169Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321170Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321171Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321172Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321173Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321175Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321176Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321177Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321178Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321179Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321182Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321183Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321184Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321185Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321186Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321187Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321188Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321189Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321190Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321191Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321192Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321193Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321194Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321195Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321196Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321197Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321203Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321204Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321205Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321206Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321207Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321208Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321209Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321210Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321211Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321212Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321213Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321214Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321215Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321216Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321217Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321218Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321219Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321220Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321221Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321222Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321223Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321224Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321225Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321226Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321227Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321228Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321229Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321230Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321232Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321233Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321234Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335321235Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2560851') ('4234', '20313900', '1335305597', None, None, None, None, None, '2560851') ('4234', None, None, None, None, None, None, None, '2560851') ('4234', '20313900', '1335305578', None, None, None, None, None, '2560851') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 521 time used for this insertion : 0.13472533226013184 save_final save missing photos in datou_result : time spend for datou_step_exec : 48.79729652404785 time spend to save output : 0.1410994529724121 total time spend for step 5 : 48.938395977020264 step6:thcl Fri Feb 7 00:21:57 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! we are using the classfication for only one thcl 3237 time to import caffe and check if the image exist : 0.007037639617919922 time to convert the images to numpy array : 0.03405332565307617 time to import caffe and check if the image exist : 0.011993169784545898 time to convert the images to numpy array : 0.04700732231140137 time to import caffe and check if the image exist : 0.014186620712280273 time to convert the images to numpy array : 0.050778865814208984 time to import caffe and check if the image exist : 0.016580581665039062 time to convert the images to numpy array : 0.0566558837890625 time to import caffe and check if the image exist : 0.020069122314453125 time to convert the images to numpy array : 0.058696746826171875 time to import caffe and check if the image exist : 0.023627519607543945 time to convert the images to numpy array : 0.05633878707885742 time to import caffe and check if the image exist : 0.02112603187561035 time to convert the images to numpy array : 0.05995917320251465 time to import caffe and check if the image exist : 0.024906158447265625 time to convert the images to numpy array : 0.058104753494262695 time to import caffe and check if the image exist : 0.02579188346862793 time to convert the images to numpy array : 0.05754399299621582 time to import caffe and check if the image exist : 0.01904773712158203 time to convert the images to numpy array : 0.06440377235412598 total time to convert the images to numpy array : 0.3258991241455078 list photo_ids error: [] list photo_ids correct : [1335321224, 1335321225, 1335321226, 1335321227, 1335321228, 1335321229, 1335321230, 1335321232, 1335321233, 1335321234, 1335321235, 1335321104, 1335321105, 1335321106, 1335321107, 1335321108, 1335321109, 1335321110, 1335321111, 1335321112, 1335321113, 1335321114, 1335321115, 1335321116, 1335321117, 1335321118, 1335321119, 1335321120, 1335321121, 1335321206, 1335321207, 1335321208, 1335321209, 1335321210, 1335321211, 1335321212, 1335321213, 1335321214, 1335321215, 1335321216, 1335321217, 1335321218, 1335321219, 1335321220, 1335321221, 1335321222, 1335321223, 1335321122, 1335321123, 1335321124, 1335321125, 1335321126, 1335321127, 1335321128, 1335321129, 1335321130, 1335321132, 1335321133, 1335321134, 1335321135, 1335321136, 1335321137, 1335321138, 1335321139, 1335321140, 1335321141, 1335321142, 1335321145, 1335321146, 1335321147, 1335321148, 1335321149, 1335321150, 1335321151, 1335321152, 1335321153, 1335321154, 1335321155, 1335321156, 1335321157, 1335321158, 1335321159, 1335321160, 1335321049, 1335321050, 1335321051, 1335321052, 1335321053, 1335321054, 1335321055, 1335321056, 1335321057, 1335321058, 1335321059, 1335321060, 1335321061, 1335321062, 1335321063, 1335321064, 1335321065, 1335321066, 1335321161, 1335321162, 1335321163, 1335321164, 1335321165, 1335321166, 1335321168, 1335321169, 1335321170, 1335321171, 1335321172, 1335321173, 1335321175, 1335321176, 1335321177, 1335321178, 1335321179, 1335321182, 1335321085, 1335321086, 1335321087, 1335321088, 1335321089, 1335321090, 1335321091, 1335321092, 1335321093, 1335321094, 1335321095, 1335321096, 1335321097, 1335321098, 1335321099, 1335321100, 1335321101, 1335321102, 1335321067, 1335321068, 1335321069, 1335321070, 1335321071, 1335321072, 1335321073, 1335321074, 1335321075, 1335321076, 1335321077, 1335321078, 1335321079, 1335321080, 1335321081, 1335321082, 1335321083, 1335321084, 1335321183, 1335321184, 1335321185, 1335321186, 1335321187, 1335321188, 1335321189, 1335321190, 1335321191, 1335321192, 1335321193, 1335321194, 1335321195, 1335321196, 1335321197, 1335321203, 1335321204, 1335321205] number of photos to traite : 173 try to delete the photos incorrect in DB tagging for thcl : 3237 To do loadFromThcl(), then load ParamDescType : thcl3237 thcls : [{'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'}] thcl {'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'} Update svm_hashtag_type_desc : 5561 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) To loadFromThcl() : net_5561 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 6469 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_rubbia_refus_2500 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_rubbia_refus_2500 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_rubbia_refus_2500 /data/models_weight/learn_rubbia_refus_2500/caffemodel size_local : 94358479 size in s3 : 94358479 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:16 caffemodel already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt size_local : 32544 size in s3 : 32544 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:15 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:09:52 mean.npy already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/synset_words.txt size_local : 334 size in s3 : 334 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:10:06 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt caffemodel_filename : /data/models_weight/learn_rubbia_refus_2500/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 6250 max_wait_temp : 1 max_wait : 0 dict_keys(['prob', 'pool5']) time used to do the prepocess of the images : 0.7745194435119629 time used to do the prediction : 0.6717820167541504 save descriptor for thcl : 3237 time to traite the descriptors : 1.0063893795013428 Catched exception ! Connect or reconnect ! storage_type for insertDescriptorsMulti : 3 To insert : 1335321224 To insert : 1335321225 To insert : 1335321226 To insert : 1335321227 To insert : 1335321228 To insert : 1335321229 To insert : 1335321230 To insert : 1335321232 To insert : 1335321233 To insert : 1335321234 To insert : 1335321235 To insert : 1335321104 To insert : 1335321105 To insert : 1335321106 To insert : 1335321107 To insert : 1335321108 To insert : 1335321109 To insert : 1335321110 To insert : 1335321111 To insert : 1335321112 To insert : 1335321113 To insert : 1335321114 To insert : 1335321115 To insert : 1335321116 To insert : 1335321117 To insert : 1335321118 To insert : 1335321119 To insert : 1335321120 To insert : 1335321121 To insert : 1335321206 To insert : 1335321207 To insert : 1335321208 To insert : 1335321209 To insert : 1335321210 To insert : 1335321211 To insert : 1335321212 To insert : 1335321213 To insert : 1335321214 To insert : 1335321215 To insert : 1335321216 To insert : 1335321217 To insert : 1335321218 To insert : 1335321219 To insert : 1335321220 To insert : 1335321221 To insert : 1335321222 To insert : 1335321223 To insert : 1335321122 To insert : 1335321123 To insert : 1335321124 To insert : 1335321125 To insert : 1335321126 To insert : 1335321127 To insert : 1335321128 To insert : 1335321129 To insert : 1335321130 To insert : 1335321132 To insert : 1335321133 To insert : 1335321134 To insert : 1335321135 To insert : 1335321136 To insert : 1335321137 To insert : 1335321138 To insert : 1335321139 To insert : 1335321140 To insert : 1335321141 To insert : 1335321142 To insert : 1335321145 To insert : 1335321146 To insert : 1335321147 To insert : 1335321148 To insert : 1335321149 To insert : 1335321150 To insert : 1335321151 To insert : 1335321152 To insert : 1335321153 To insert : 1335321154 To insert : 1335321155 To insert : 1335321156 To insert : 1335321157 To insert : 1335321158 To insert : 1335321159 To insert : 1335321160 To insert : 1335321049 To insert : 1335321050 To insert : 1335321051 To insert : 1335321052 To insert : 1335321053 To insert : 1335321054 To insert : 1335321055 To insert : 1335321056 To insert : 1335321057 To insert : 1335321058 To insert : 1335321059 To insert : 1335321060 To insert : 1335321061 To insert : 1335321062 To insert : 1335321063 To insert : 1335321064 To insert : 1335321065 To insert : 1335321066 To insert : 1335321161 To insert : 1335321162 To insert : 1335321163 To insert : 1335321164 To insert : 1335321165 To insert : 1335321166 To insert : 1335321168 To insert : 1335321169 To insert : 1335321170 To insert : 1335321171 To insert : 1335321172 To insert : 1335321173 To insert : 1335321175 To insert : 1335321176 To insert : 1335321177 To insert : 1335321178 To insert : 1335321179 To insert : 1335321182 To insert : 1335321085 To insert : 1335321086 To insert : 1335321087 To insert : 1335321088 To insert : 1335321089 To insert : 1335321090 To insert : 1335321091 To insert : 1335321092 To insert : 1335321093 To insert : 1335321094 To insert : 1335321095 To insert : 1335321096 To insert : 1335321097 To insert : 1335321098 To insert : 1335321099 To insert : 1335321100 To insert : 1335321101 To insert : 1335321102 To insert : 1335321067 To insert : 1335321068 To insert : 1335321069 To insert : 1335321070 To insert : 1335321071 To insert : 1335321072 To insert : 1335321073 To insert : 1335321074 To insert : 1335321075 To insert : 1335321076 To insert : 1335321077 To insert : 1335321078 To insert : 1335321079 To insert : 1335321080 To insert : 1335321081 To insert : 1335321082 To insert : 1335321083 To insert : 1335321084 To insert : 1335321183 To insert : 1335321184 To insert : 1335321185 To insert : 1335321186 To insert : 1335321187 To insert : 1335321188 To insert : 1335321189 To insert : 1335321190 To insert : 1335321191 To insert : 1335321192 To insert : 1335321193 To insert : 1335321194 To insert : 1335321195 To insert : 1335321196 To insert : 1335321197 To insert : 1335321203 To insert : 1335321204 To insert : 1335321205 time to insert the descriptors : 29.635746717453003 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos SAVE THCL : begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 259 time used for this insertion : 0.021093368530273438 save missing photos in datou_result : time spend for datou_step_exec : 36.36503624916077 time spend to save output : 0.0660104751586914 total time spend for step 6 : 36.43104672431946 step7:ventilate_hashtags_in_portfolio Fri Feb 7 00:22:34 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 20313900 get user id for portfolio 20313900 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20313900 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Tetrapak','Papier','PET_clair','refus','PEHD','carton','pehd','papier','pet_clair','metal','Film_plastique','PET_fonce','environnement','Carton','pet_fonce','flou','mal_croppe','autre')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20313900 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Tetrapak','Papier','PET_clair','refus','PEHD','carton','pehd','papier','pet_clair','metal','Film_plastique','PET_fonce','environnement','Carton','pet_fonce','flou','mal_croppe','autre')) AND mptpi.`min_score`=0.6 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20313900 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Tetrapak','Papier','PET_clair','refus','PEHD','carton','pehd','papier','pet_clair','metal','Film_plastique','PET_fonce','environnement','Carton','pet_fonce','flou','mal_croppe','autre')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20313900 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Tetrapak','Papier','PET_clair','refus','PEHD','carton','pehd','papier','pet_clair','metal','Film_plastique','PET_fonce','environnement','Carton','pet_fonce','flou','mal_croppe','autre')) AND mptpi.`min_score`=0.6 To do lien utilise dans velours : https://www.fotonower.com/velours/20314067,20314074,20314075,20314070,20314073,20314080,20314076,20314077,20314081,20314079,20314082,20314083,20314084?tags=tetrapak,papier,pet_clair,refus,pehd,carton,metal,film_plastique,pet_fonce,environnement,flou,mal_croppe,autre&datou_id_consolidate=4235&port_consolidate=20313900 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1335305597, 1335305578] Looping around the photos to save general results len do output : 1 /20313900. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2560851') ('4234', '20313900', '1335305597', None, None, None, None, None, '2560851') ('4234', None, None, None, None, None, None, None, '2560851') ('4234', '20313900', '1335305578', None, None, None, None, None, '2560851') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.014081001281738281 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.2476181983947754 time spend to save output : 0.014324665069580078 total time spend for step 7 : 2.2619428634643555 step8:final Fri Feb 7 00:22:36 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1335305597: ('0.233083818434412',), 1335305578: ('0.233083818434412',)} new output for save of step final : {1335305597: ('0.233083818434412',), 1335305578: ('0.233083818434412',)} [1335305597, 1335305578] Looping around the photos to save general results len do output : 2 /1335305597.Didn't retrieve data . /1335305578.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2560851') ('4234', '20313900', '1335305597', None, None, None, None, None, '2560851') ('4234', None, None, None, None, None, None, None, '2560851') ('4234', '20313900', '1335305578', None, None, None, None, None, '2560851') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.012885570526123047 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.11478877067565918 time spend to save output : 0.013075828552246094 total time spend for step 8 : 0.12786459922790527 step9:velours_tree Fri Feb 7 00:22:36 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure list_portfolios : 20314067,20314074,20314075,20314070,20314073,20314080,20314076,20314077,20314081,20314079,20314082,20314083,20314084 photo desc type : 5561 - Retrieving photos to tag... query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20314067 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20314074 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20314075 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20314070 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20314073 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20314080 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20314076 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20314077 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20314081 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20314079 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20314082 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20314083 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20314084 ORDER BY ph.size desc - Loading descriptors... Size : 2048 len(descriptors) : 173 Compute structured hierarchical clustering... ward : AgglomerativeClustering(n_clusters=6) ward.labels_ : [5 0 0 1 4 4 1 0 0 0 0 0 0 1 1 0 1 0 0 0 0 0 0 1 0 1 0 4 1 0 0 0 5 5 4 2 0 5 0 0 1 1 5 0 0 0 0 0 1 4 0 4 0 1 0 1 0 0 0 5 0 1 0 1 0 0 0 0 0 0 5 0 0 1 0 1 0 0 0 1 0 0 0 5 2 0 0 1 1 0 0 4 3 4 1 1 0 0 4 0 0 2 0 0 3 3 1 3 4 1 0 0 0 0 0 0 2 0 0 0 3 0 1 1 5 4 4 4 0 4 2 4 5 5 5 4 4 4 4 2 5 3 0 0 2 0 0 0 0 4 5 5 5 0 5 5 3 0 5 4 2 0 0 0 0 0 0 0 3 5 1 5 5] Elapsed time: 0.027601957321166992 graph_id used : 76618 - Beta version, working pretty good on 11-5-16 ! https://www.fotonower.com/velours/20314067,20314074,20314075,20314070,20314073,20314080,20314076,20314077,20314081,20314079,20314082,20314083,20314084?tags=tetrapak,papier,pet_clair,refus,pehd,carton,metal,film_plastique,pet_fonce,environnement,flou,mal_croppe,autre&datou_id_consolidate=4235&port_consolidate=20313900&tree_id=76618 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : velours_tree we use saveGeneral [1335305597, 1335305578] Looping around the photos to save general results len do output : 1 /20313900Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2560851') ('4234', '20313900', '1335305597', None, None, None, None, None, '2560851') ('4234', None, None, None, None, None, None, None, '2560851') ('4234', '20313900', '1335305578', None, None, None, None, None, '2560851') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.012903213500976562 save_final save missing photos in datou_result : time spend for datou_step_exec : 11.266889095306396 time spend to save output : 0.013288497924804688 total time spend for step 9 : 11.280177593231201 step10:send_mail_cod Fri Feb 7 00:22:47 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Qualipapia_P20313900_07-02-2025_00_22_47.pdf 20314049 imagette203140491738884167 20314056 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203140561738884167 20314057 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203140571738884168 20314052 imagette203140521738884169 20314055 imagette203140551738884169 20314062 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203140621738884169 20314058 change filename to text .change filename to text .imagette203140581738884170 20314059 imagette203140591738884170 20314063 imagette203140631738884170 20314064 imagette203140641738884170 20314065 imagette203140651738884170 20314066 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203140661738884170 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20313900 and hashtag_type = 4230 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20314067,20314074,20314075,20314070,20314073,20314080,20314076,20314077,20314081,20314079,20314082,20314083,20314084?tags=tetrapak,papier,pet_clair,refus,pehd,carton,metal,film_plastique,pet_fonce,environnement,flou,mal_croppe,autre&datou_id_consolidate=4235&port_consolidate=20313900&tree_id=76618 your option no_mail is active, we will not send the real mail to your client args[1335305597] : ((1335305597, -7.787097480236104, 492609224), (1335305597, -0.34306000937327946, 496442774), '0.233083818434412') apple ((1335305597, -7.787097480236104, 492609224), (1335305597, -0.34306000937327946, 496442774), '0.233083818434412') We are sending mail with results at cod@fotonower.com args[1335305578] : ((1335305578, -7.767715566346525, 492609224), (1335305578, -0.36129681398869157, 496442774), '0.233083818434412') apple ((1335305578, -7.767715566346525, 492609224), (1335305578, -0.36129681398869157, 496442774), '0.233083818434412') We are sending mail with results at cod@fotonower.com refus_total : 0.233083818434412 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20313900 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20313900_07-02-2025_00_22_47.pdf results_Qualipapia_P20313900_07-02-2025_00_22_47.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20313900_07-02-2025_00_22_47.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4234','20313900','results_Qualipapia_P20313900_07-02-2025_00_22_47.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20313900_07-02-2025_00_22_47.pdf','pdf','','0.16','0.233083818434412') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1335305597, 1335305578] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2560851') ('4234', '20313900', '1335305597', None, None, None, None, None, '2560851') ('4234', None, None, None, None, None, None, None, '2560851') ('4234', '20313900', '1335305578', None, None, None, None, None, '2560851') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 2 time used for this insertion : 0.012062311172485352 save_final save missing photos in datou_result : time spend for datou_step_exec : 5.533389329910278 time spend to save output : 0.012194395065307617 total time spend for step 10 : 5.545583724975586 step11:split_time_score Fri Feb 7 00:22:53 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 3442, 'mtr_user_id': 31, 'name': 'classifieur_2camions_valcor_021122_v1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'deux_camions,camion_droite,camion_gauche,pas_de_camion', 'svm_portfolios_learning': '7659379,7659034,7657685,7657114', 'photo_hashtag_type': 4458, 'photo_desc_type': 5723, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107760533,2107760534,2107760535,2107760536'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('17', 2),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 06022025 20313900 Nombre de photos uploadées : 2 / 23040 (0%) 06022025 20313900 Nombre de photos taguées (types de déchets): 2 / 2 (100%) 06022025 20313900 Nombre de photos taguées (volume) : 0 / 2 (0%) elapsed_time : load_data_split_time_score 9.5367431640625e-07 elapsed_time : order_list_meta_photo_and_scores 4.0531158447265625e-06 ?? elapsed_time : fill_and_build_computed_from_old_data 0.00015807151794433594 elapsed_time : insert_dashboard_record_day_entry 0.021538734436035156 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20297627 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20297627 order by id desc limit 1 Qualite : 0.1437886216255483 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20297628_06-02-2025_13_32_26.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20297628 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20297628 AND mptpi.`type`=4230 To do Qualite : 0.1437886216255483 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20297628_06-02-2025_13_32_26.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20297628 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20297628 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20297629 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20298032 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20298033 order by id desc limit 1 Qualite : 0.1283856179969406 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20298034_06-02-2025_17_41_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20298034 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20298034 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20298035 order by id desc limit 1 Qualite : 0.15398963015225456 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20298036_06-02-2025_18_00_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20298036 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20298036 AND mptpi.`type`=4230 To do Qualite : 0.23439323561320993 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20300744_06-02-2025_18_42_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20300744 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20300744 AND mptpi.`type`=4230 To do Qualite : 0.21397048530398904 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20300745_06-02-2025_18_20_24.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20300745 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20300745 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20305751 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20305752 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20305753 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20305754 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20305755 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20305780 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20305781 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20305782 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20305783 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20305784 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20305785 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20305786 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20305787 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20305788 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20305789 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20305790 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20311741 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20311742 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20309441 order by id desc limit 1 Qualite : 0.21829823758037809 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20305792_06-02-2025_20_38_42.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20305792 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20305792 AND mptpi.`type`=4230 To do Qualite : 0.24527143931636783 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20305793_06-02-2025_20_21_33.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20305793 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20305793 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20309442 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20309443 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20309444 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20309445 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20309446 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20309447 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20309448 order by id desc limit 1 Qualite : 0.20351072599550035 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20307171_06-02-2025_20_48_02.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20307171 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20307171 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20307172 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20308356 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20308357 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20308358 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20308359 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20308360 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20308361 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20308362 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20308363 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20308364 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20308365 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20308366 order by id desc limit 1 Qualite : 0.186778754333038 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20308367_06-02-2025_21_35_29.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20308367 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20308367 AND mptpi.`type`=4230 To do Qualite : 0.18142782694266635 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20308368_06-02-2025_21_25_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20308368 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20308368 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20309449 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20309450 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20309451 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20309452 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20309453 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20309454 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20309455 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20309456 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20309457 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20309458 order by id desc limit 1 Qualite : 0.17501760183289164 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20309459_06-02-2025_21_47_10.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20309459 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20309459 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20311743 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20311744 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20311745 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20310713 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20310714 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20310715 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20310716 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20310717 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20310718 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20310719 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20310720 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20310721 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20310722 order by id desc limit 1 Qualite : 0.18614081699614712 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20310723_06-02-2025_22_46_35.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20310723 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20310723 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20310724 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20311768 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20311783 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20311784 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20311785 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20311786 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20311787 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20311788 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20311789 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20311790 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20311791 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20311792 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20311793 order by id desc limit 1 Qualite : 0.20198839558319584 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20311794_06-02-2025_23_51_22.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20311794 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20311794 AND mptpi.`type`=4230 To do Qualite : 0.23592222524593415 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20311795_06-02-2025_23_34_07.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20311795 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20311795 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20313881 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20313883 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20313885 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20313886 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20313887 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20313888 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20313889 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20313890 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20313892 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20313894 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20313895 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20313896 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20313897 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20313898 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20313899 order by id desc limit 1 Qualite : 0.233083818434412 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20313900_07-02-2025_00_22_47.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20313900 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20313900 AND mptpi.`type`=4230 To do Qualite : 0.2386101662580131 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20313901_07-02-2025_00_19_48.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20313901 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20313901 AND mptpi.`type`=4230 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'06022025': {'nb_upload': 2, 'nb_taggue_class': 2, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1335305597, 1335305578] Looping around the photos to save general results len do output : 1 /20313900Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2560851') ('4234', '20313900', '1335305597', None, None, None, None, None, '2560851') ('4234', None, None, None, None, None, None, None, '2560851') ('4234', '20313900', '1335305578', None, None, None, None, None, '2560851') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.012729883193969727 save_final save missing photos in datou_result : time spend for datou_step_exec : 74.11949467658997 time spend to save output : 0.01301431655883789 total time spend for step 11 : 74.1325089931488 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 1 len(input) + len(total_photo_id_missing) : 2 set_done_treatment 35.72user 25.37system 3:42.86elapsed 27%CPU (0avgtext+0avgdata 2569016maxresident)k 698256inputs+12992outputs (36major+2396454minor)pagefaults 0swaps