python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 4234 -l 5 ' -s datou_current_4234 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 4155621 load datou : 4234 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load THCL from format json or kwargs add thcl : 3237 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig add pdt : 5561 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4234, datou_cur_ids : ['2579746'] with mtr_portfolio_ids : ['20479443'] and first list_photo_ids : [] new path : /proc/4155621/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, brightness, blur_detection, rle_unique_nms_with_priority, crop_condition, thcl, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , BFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 2 ; length of list_pids : 2 ; length of list_args : 2 time to download the photos : 0.2542729377746582 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Wed Feb 12 16:20:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10332 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-12 16:20:33.644485: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-12 16:20:33.653753: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-12 16:20:33.655621: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f5b14000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-12 16:20:33.655692: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-12 16:20:33.658887: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-12 16:20:33.884239: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1e1ac800 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-12 16:20:33.884306: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-12 16:20:33.885932: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-12 16:20:33.886526: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-12 16:20:33.889890: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-12 16:20:33.893029: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-12 16:20:33.893600: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-12 16:20:33.896737: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-12 16:20:33.898150: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-12 16:20:33.904123: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-12 16:20:33.905828: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-12 16:20:33.905955: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-12 16:20:33.906837: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-12 16:20:33.906858: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-12 16:20:33.906867: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-12 16:20:33.908417: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9570 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-12 16:20:34.232719: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-12 16:20:34.232863: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-12 16:20:34.232892: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-12 16:20:34.232916: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-12 16:20:34.232941: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-12 16:20:34.232964: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-12 16:20:34.232986: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-12 16:20:34.233008: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-12 16:20:34.234615: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-12 16:20:34.236198: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-12 16:20:34.236310: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-12 16:20:34.236336: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-12 16:20:34.236360: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-12 16:20:34.236385: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-12 16:20:34.236407: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-12 16:20:34.236430: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-12 16:20:34.236452: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-12 16:20:34.238098: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-12 16:20:34.238161: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-12 16:20:34.238175: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-12 16:20:34.238187: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-12 16:20:34.239876: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9570 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-12 16:20:42.080524: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-12 16:20:42.279827: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 2 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -115.09844 max: 148.76406 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 0.00011610984802246094 nb_pixel_total : 830 time to create 1 rle with old method : 0.0018379688262939453 length of segment : 39 time for calcul the mask position with numpy : 6.103515625e-05 nb_pixel_total : 14 time to create 1 rle with old method : 7.2479248046875e-05 length of segment : 10 time for calcul the mask position with numpy : 7.128715515136719e-05 nb_pixel_total : 290 time to create 1 rle with old method : 0.0005726814270019531 length of segment : 34 time for calcul the mask position with numpy : 7.510185241699219e-05 nb_pixel_total : 268 time to create 1 rle with old method : 0.0005595684051513672 length of segment : 26 time for calcul the mask position with numpy : 6.389617919921875e-05 nb_pixel_total : 37 time to create 1 rle with old method : 0.00018548965454101562 length of segment : 5 time for calcul the mask position with numpy : 7.43865966796875e-05 nb_pixel_total : 102 time to create 1 rle with old method : 0.00028896331787109375 length of segment : 23 time for calcul the mask position with numpy : 0.00043487548828125 nb_pixel_total : 4581 time to create 1 rle with old method : 0.008399009704589844 length of segment : 159 time for calcul the mask position with numpy : 0.00011730194091796875 nb_pixel_total : 769 time to create 1 rle with old method : 0.001422882080078125 length of segment : 73 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -119.30938 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 14 time for calcul the mask position with numpy : 7.152557373046875e-05 nb_pixel_total : 356 time to create 1 rle with old method : 0.0005018711090087891 length of segment : 49 time for calcul the mask position with numpy : 6.604194641113281e-05 nb_pixel_total : 436 time to create 1 rle with old method : 0.0009388923645019531 length of segment : 25 time for calcul the mask position with numpy : 0.0002841949462890625 nb_pixel_total : 9823 time to create 1 rle with old method : 0.011770486831665039 length of segment : 157 time for calcul the mask position with numpy : 6.341934204101562e-05 nb_pixel_total : 440 time to create 1 rle with old method : 0.0005712509155273438 length of segment : 45 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 408 time to create 1 rle with old method : 0.00057220458984375 length of segment : 33 time for calcul the mask position with numpy : 4.744529724121094e-05 nb_pixel_total : 439 time to create 1 rle with old method : 0.0006196498870849609 length of segment : 36 time for calcul the mask position with numpy : 4.863739013671875e-05 nb_pixel_total : 671 time to create 1 rle with old method : 0.0009872913360595703 length of segment : 43 time for calcul the mask position with numpy : 4.76837158203125e-05 nb_pixel_total : 710 time to create 1 rle with old method : 0.0010433197021484375 length of segment : 32 time for calcul the mask position with numpy : 6.103515625e-05 nb_pixel_total : 170 time to create 1 rle with old method : 0.00047779083251953125 length of segment : 10 time for calcul the mask position with numpy : 4.935264587402344e-05 nb_pixel_total : 628 time to create 1 rle with old method : 0.0008339881896972656 length of segment : 44 time for calcul the mask position with numpy : 5.435943603515625e-05 nb_pixel_total : 680 time to create 1 rle with old method : 0.0009546279907226562 length of segment : 43 time for calcul the mask position with numpy : 7.2479248046875e-05 nb_pixel_total : 1074 time to create 1 rle with old method : 0.0014710426330566406 length of segment : 57 time for calcul the mask position with numpy : 5.5789947509765625e-05 nb_pixel_total : 851 time to create 1 rle with old method : 0.001178741455078125 length of segment : 39 time for calcul the mask position with numpy : 6.747245788574219e-05 nb_pixel_total : 924 time to create 1 rle with old method : 0.0013246536254882812 length of segment : 45 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -113.96172 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 0.00012493133544921875 nb_pixel_total : 6216 time to create 1 rle with old method : 0.009202957153320312 length of segment : 122 time for calcul the mask position with numpy : 5.459785461425781e-05 nb_pixel_total : 769 time to create 1 rle with old method : 0.0013895034790039062 length of segment : 31 time for calcul the mask position with numpy : 0.0001373291015625 nb_pixel_total : 4660 time to create 1 rle with old method : 0.005897998809814453 length of segment : 93 time for calcul the mask position with numpy : 8.559226989746094e-05 nb_pixel_total : 332 time to create 1 rle with old method : 0.0009059906005859375 length of segment : 22 time for calcul the mask position with numpy : 0.00010228157043457031 nb_pixel_total : 1210 time to create 1 rle with old method : 0.001733541488647461 length of segment : 20 time for calcul the mask position with numpy : 0.0004391670227050781 nb_pixel_total : 19411 time to create 1 rle with old method : 0.025277137756347656 length of segment : 171 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -121.86406 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 0.00018262863159179688 nb_pixel_total : 8302 time to create 1 rle with old method : 0.012662649154663086 length of segment : 125 time for calcul the mask position with numpy : 6.175041198730469e-05 nb_pixel_total : 462 time to create 1 rle with old method : 0.0007021427154541016 length of segment : 22 time for calcul the mask position with numpy : 6.699562072753906e-05 nb_pixel_total : 1156 time to create 1 rle with old method : 0.0013930797576904297 length of segment : 60 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 220 time to create 1 rle with old method : 0.00032210350036621094 length of segment : 22 time for calcul the mask position with numpy : 5.316734313964844e-05 nb_pixel_total : 643 time to create 1 rle with old method : 0.0008299350738525391 length of segment : 48 time for calcul the mask position with numpy : 3.4809112548828125e-05 nb_pixel_total : 101 time to create 1 rle with old method : 0.00018978118896484375 length of segment : 7 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 235.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 112.40859 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 6.794929504394531e-05 nb_pixel_total : 472 time to create 1 rle with old method : 0.0009620189666748047 length of segment : 19 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 433 time to create 1 rle with old method : 0.0006604194641113281 length of segment : 19 time for calcul the mask position with numpy : 0.0004968643188476562 nb_pixel_total : 38124 time to create 1 rle with old method : 0.04324221611022949 length of segment : 290 time for calcul the mask position with numpy : 0.0009493827819824219 nb_pixel_total : 46504 time to create 1 rle with old method : 0.05225110054016113 length of segment : 309 Processing 1 images image shape: (400, 400, 3) min: 7.00000 max: 213.00000 molded_images shape: (1, 640, 640, 3) min: -89.24375 max: 82.67656 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -111.82891 max: 147.36953 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 9.226799011230469e-05 nb_pixel_total : 1212 time to create 1 rle with old method : 0.001725912094116211 length of segment : 27 time for calcul the mask position with numpy : 5.412101745605469e-05 nb_pixel_total : 393 time to create 1 rle with old method : 0.0006504058837890625 length of segment : 19 time for calcul the mask position with numpy : 9.560585021972656e-05 nb_pixel_total : 2875 time to create 1 rle with old method : 0.0037279129028320312 length of segment : 50 time for calcul the mask position with numpy : 5.412101745605469e-05 nb_pixel_total : 188 time to create 1 rle with old method : 0.00031685829162597656 length of segment : 13 time for calcul the mask position with numpy : 6.556510925292969e-05 nb_pixel_total : 793 time to create 1 rle with old method : 0.0011265277862548828 length of segment : 37 time for calcul the mask position with numpy : 0.0003108978271484375 nb_pixel_total : 9216 time to create 1 rle with old method : 0.011128425598144531 length of segment : 155 time for calcul the mask position with numpy : 0.00025582313537597656 nb_pixel_total : 5783 time to create 1 rle with old method : 0.00679469108581543 length of segment : 206 time for calcul the mask position with numpy : 6.842613220214844e-05 nb_pixel_total : 810 time to create 1 rle with old method : 0.0010423660278320312 length of segment : 31 time for calcul the mask position with numpy : 5.125999450683594e-05 nb_pixel_total : 738 time to create 1 rle with old method : 0.0009489059448242188 length of segment : 29 Processing 1 images image shape: (400, 400, 3) min: 4.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -116.51250 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 11 time for calcul the mask position with numpy : 6.914138793945312e-05 nb_pixel_total : 778 time to create 1 rle with old method : 0.0011107921600341797 length of segment : 33 time for calcul the mask position with numpy : 4.7206878662109375e-05 nb_pixel_total : 413 time to create 1 rle with old method : 0.0006935596466064453 length of segment : 15 time for calcul the mask position with numpy : 8.749961853027344e-05 nb_pixel_total : 1089 time to create 1 rle with old method : 0.0020599365234375 length of segment : 48 time for calcul the mask position with numpy : 0.00011110305786132812 nb_pixel_total : 4756 time to create 1 rle with old method : 0.006844758987426758 length of segment : 90 time for calcul the mask position with numpy : 4.982948303222656e-05 nb_pixel_total : 245 time to create 1 rle with old method : 0.00037288665771484375 length of segment : 24 time for calcul the mask position with numpy : 6.556510925292969e-05 nb_pixel_total : 1078 time to create 1 rle with old method : 0.0013599395751953125 length of segment : 50 time for calcul the mask position with numpy : 4.863739013671875e-05 nb_pixel_total : 241 time to create 1 rle with old method : 0.00042891502380371094 length of segment : 22 time for calcul the mask position with numpy : 5.340576171875e-05 nb_pixel_total : 744 time to create 1 rle with old method : 0.0009298324584960938 length of segment : 31 time for calcul the mask position with numpy : 0.00011754035949707031 nb_pixel_total : 4795 time to create 1 rle with old method : 0.005755901336669922 length of segment : 91 time for calcul the mask position with numpy : 0.0001418590545654297 nb_pixel_total : 5022 time to create 1 rle with old method : 0.006018400192260742 length of segment : 99 time for calcul the mask position with numpy : 0.00013256072998046875 nb_pixel_total : 5137 time to create 1 rle with old method : 0.0059680938720703125 length of segment : 90 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -121.75859 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 7.748603820800781e-05 nb_pixel_total : 1850 time to create 1 rle with old method : 0.002252817153930664 length of segment : 125 time for calcul the mask position with numpy : 9.918212890625e-05 nb_pixel_total : 1538 time to create 1 rle with old method : 0.001954793930053711 length of segment : 59 time for calcul the mask position with numpy : 0.00021767616271972656 nb_pixel_total : 1419 time to create 1 rle with old method : 0.001968860626220703 length of segment : 164 time for calcul the mask position with numpy : 0.00046181678771972656 nb_pixel_total : 19601 time to create 1 rle with old method : 0.028536081314086914 length of segment : 293 time for calcul the mask position with numpy : 5.817413330078125e-05 nb_pixel_total : 59 time to create 1 rle with old method : 0.0001728534698486328 length of segment : 20 time for calcul the mask position with numpy : 7.486343383789062e-05 nb_pixel_total : 2336 time to create 1 rle with old method : 0.0029053688049316406 length of segment : 71 time for calcul the mask position with numpy : 6.103515625e-05 nb_pixel_total : 1253 time to create 1 rle with old method : 0.0015625953674316406 length of segment : 42 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 121.77187 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 9.202957153320312e-05 nb_pixel_total : 889 time to create 1 rle with old method : 0.0015828609466552734 length of segment : 36 time for calcul the mask position with numpy : 6.0558319091796875e-05 nb_pixel_total : 365 time to create 1 rle with old method : 0.0007600784301757812 length of segment : 38 time for calcul the mask position with numpy : 5.1975250244140625e-05 nb_pixel_total : 740 time to create 1 rle with old method : 0.0009703636169433594 length of segment : 42 time for calcul the mask position with numpy : 4.792213439941406e-05 nb_pixel_total : 490 time to create 1 rle with old method : 0.0007195472717285156 length of segment : 33 time for calcul the mask position with numpy : 0.00014448165893554688 nb_pixel_total : 1298 time to create 1 rle with old method : 0.0017478466033935547 length of segment : 141 time for calcul the mask position with numpy : 0.00012063980102539062 nb_pixel_total : 478 time to create 1 rle with old method : 0.0009889602661132812 length of segment : 67 time for calcul the mask position with numpy : 0.00026798248291015625 nb_pixel_total : 3787 time to create 1 rle with old method : 0.006137371063232422 length of segment : 191 time for calcul the mask position with numpy : 0.0003311634063720703 nb_pixel_total : 3402 time to create 1 rle with old method : 0.007220745086669922 length of segment : 181 Processing 1 images image shape: (280, 400, 3) min: 2.00000 max: 206.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 65.37578 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 146.00625 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 14 time for calcul the mask position with numpy : 9.441375732421875e-05 nb_pixel_total : 3452 time to create 1 rle with old method : 0.004084110260009766 length of segment : 60 time for calcul the mask position with numpy : 4.649162292480469e-05 nb_pixel_total : 780 time to create 1 rle with old method : 0.0010178089141845703 length of segment : 43 time for calcul the mask position with numpy : 0.00014925003051757812 nb_pixel_total : 2940 time to create 1 rle with old method : 0.004588127136230469 length of segment : 82 time for calcul the mask position with numpy : 4.5299530029296875e-05 nb_pixel_total : 274 time to create 1 rle with old method : 0.00036525726318359375 length of segment : 30 time for calcul the mask position with numpy : 3.910064697265625e-05 nb_pixel_total : 425 time to create 1 rle with old method : 0.0005710124969482422 length of segment : 26 time for calcul the mask position with numpy : 4.100799560546875e-05 nb_pixel_total : 480 time to create 1 rle with old method : 0.0006544589996337891 length of segment : 27 time for calcul the mask position with numpy : 4.2438507080078125e-05 nb_pixel_total : 543 time to create 1 rle with old method : 0.0007631778717041016 length of segment : 33 time for calcul the mask position with numpy : 5.125999450683594e-05 nb_pixel_total : 735 time to create 1 rle with old method : 0.0009279251098632812 length of segment : 53 time for calcul the mask position with numpy : 6.580352783203125e-05 nb_pixel_total : 1398 time to create 1 rle with old method : 0.0018706321716308594 length of segment : 70 time for calcul the mask position with numpy : 6.842613220214844e-05 nb_pixel_total : 1906 time to create 1 rle with old method : 0.0023758411407470703 length of segment : 77 time for calcul the mask position with numpy : 0.00017404556274414062 nb_pixel_total : 1356 time to create 1 rle with old method : 0.0039098262786865234 length of segment : 68 time for calcul the mask position with numpy : 0.00012946128845214844 nb_pixel_total : 668 time to create 1 rle with old method : 0.0014774799346923828 length of segment : 29 time for calcul the mask position with numpy : 0.00012540817260742188 nb_pixel_total : 1967 time to create 1 rle with old method : 0.004029273986816406 length of segment : 94 time for calcul the mask position with numpy : 9.393692016601562e-05 nb_pixel_total : 694 time to create 1 rle with old method : 0.0015764236450195312 length of segment : 30 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 148.08047 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 13 time for calcul the mask position with numpy : 4.935264587402344e-05 nb_pixel_total : 460 time to create 1 rle with old method : 0.0006198883056640625 length of segment : 22 time for calcul the mask position with numpy : 4.029273986816406e-05 nb_pixel_total : 339 time to create 1 rle with old method : 0.0004673004150390625 length of segment : 26 time for calcul the mask position with numpy : 4.100799560546875e-05 nb_pixel_total : 358 time to create 1 rle with old method : 0.0005128383636474609 length of segment : 27 time for calcul the mask position with numpy : 4.0531158447265625e-05 nb_pixel_total : 179 time to create 1 rle with old method : 0.00026798248291015625 length of segment : 21 time for calcul the mask position with numpy : 4.100799560546875e-05 nb_pixel_total : 378 time to create 1 rle with old method : 0.0005216598510742188 length of segment : 26 time for calcul the mask position with numpy : 4.5299530029296875e-05 nb_pixel_total : 595 time to create 1 rle with old method : 0.0008020401000976562 length of segment : 31 time for calcul the mask position with numpy : 4.100799560546875e-05 nb_pixel_total : 76 time to create 1 rle with old method : 0.0001556873321533203 length of segment : 6 time for calcul the mask position with numpy : 4.172325134277344e-05 nb_pixel_total : 285 time to create 1 rle with old method : 0.0004227161407470703 length of segment : 22 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 199 time to create 1 rle with old method : 0.0003192424774169922 length of segment : 13 time for calcul the mask position with numpy : 4.1484832763671875e-05 nb_pixel_total : 183 time to create 1 rle with old method : 0.0002849102020263672 length of segment : 17 time for calcul the mask position with numpy : 6.914138793945312e-05 nb_pixel_total : 384 time to create 1 rle with old method : 0.003562450408935547 length of segment : 18 time for calcul the mask position with numpy : 7.2479248046875e-05 nb_pixel_total : 430 time to create 1 rle with old method : 0.0009081363677978516 length of segment : 24 time for calcul the mask position with numpy : 3.719329833984375e-05 nb_pixel_total : 85 time to create 1 rle with old method : 0.00015997886657714844 length of segment : 11 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 149.80312 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 5.030632019042969e-05 nb_pixel_total : 545 time to create 1 rle with old method : 0.0007233619689941406 length of segment : 33 length of segment : 0 time for calcul the mask position with numpy : 3.552436828613281e-05 nb_pixel_total : 102 time to create 1 rle with old method : 0.00023245811462402344 length of segment : 11 time for calcul the mask position with numpy : 3.9577484130859375e-05 nb_pixel_total : 542 time to create 1 rle with old method : 0.0007023811340332031 length of segment : 23 time for calcul the mask position with numpy : 0.000102996826171875 nb_pixel_total : 227 time to create 1 rle with old method : 0.00032448768615722656 length of segment : 26 time for calcul the mask position with numpy : 6.914138793945312e-05 nb_pixel_total : 1260 time to create 1 rle with old method : 0.0015707015991210938 length of segment : 65 time for calcul the mask position with numpy : 0.0004630088806152344 nb_pixel_total : 12504 time to create 1 rle with old method : 0.021025896072387695 length of segment : 157 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 135.53750 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 9.036064147949219e-05 nb_pixel_total : 212 time to create 1 rle with old method : 0.0006303787231445312 length of segment : 56 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -116.88750 max: 147.93984 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 8.463859558105469e-05 nb_pixel_total : 42 time to create 1 rle with old method : 0.0001430511474609375 length of segment : 27 time for calcul the mask position with numpy : 5.5789947509765625e-05 nb_pixel_total : 847 time to create 1 rle with old method : 0.0011556148529052734 length of segment : 37 time for calcul the mask position with numpy : 5.221366882324219e-05 nb_pixel_total : 651 time to create 1 rle with old method : 0.0009362697601318359 length of segment : 31 time for calcul the mask position with numpy : 3.695487976074219e-05 nb_pixel_total : 8 time to create 1 rle with old method : 4.839897155761719e-05 length of segment : 2 time for calcul the mask position with numpy : 4.601478576660156e-05 nb_pixel_total : 267 time to create 1 rle with old method : 0.0007431507110595703 length of segment : 29 time for calcul the mask position with numpy : 7.2479248046875e-05 nb_pixel_total : 43 time to create 1 rle with old method : 0.0001590251922607422 length of segment : 15 length of segment : 0 time for calcul the mask position with numpy : 5.14984130859375e-05 nb_pixel_total : 422 time to create 1 rle with old method : 0.0006887912750244141 length of segment : 33 time for calcul the mask position with numpy : 4.792213439941406e-05 nb_pixel_total : 110 time to create 1 rle with old method : 0.00021505355834960938 length of segment : 27 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -115.87969 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 8.106231689453125e-05 nb_pixel_total : 1262 time to create 1 rle with old method : 0.002222776412963867 length of segment : 42 time for calcul the mask position with numpy : 5.984306335449219e-05 nb_pixel_total : 446 time to create 1 rle with old method : 0.0008704662322998047 length of segment : 22 time for calcul the mask position with numpy : 0.00010037422180175781 nb_pixel_total : 2527 time to create 1 rle with old method : 0.0042247772216796875 length of segment : 50 time for calcul the mask position with numpy : 6.008148193359375e-05 nb_pixel_total : 333 time to create 1 rle with old method : 0.0006597042083740234 length of segment : 17 time for calcul the mask position with numpy : 5.555152893066406e-05 nb_pixel_total : 292 time to create 1 rle with old method : 0.0005745887756347656 length of segment : 21 time for calcul the mask position with numpy : 6.413459777832031e-05 nb_pixel_total : 822 time to create 1 rle with old method : 0.001428842544555664 length of segment : 33 time for calcul the mask position with numpy : 6.270408630371094e-05 nb_pixel_total : 309 time to create 1 rle with old method : 0.0006155967712402344 length of segment : 18 time for calcul the mask position with numpy : 9.846687316894531e-05 nb_pixel_total : 2563 time to create 1 rle with old method : 0.004313468933105469 length of segment : 53 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -112.79375 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 0.00012063980102539062 nb_pixel_total : 5381 time to create 1 rle with old method : 0.0062487125396728516 length of segment : 138 time for calcul the mask position with numpy : 8.726119995117188e-05 nb_pixel_total : 1394 time to create 1 rle with old method : 0.0019207000732421875 length of segment : 23 time for calcul the mask position with numpy : 0.00013756752014160156 nb_pixel_total : 3624 time to create 1 rle with old method : 0.004405498504638672 length of segment : 101 time for calcul the mask position with numpy : 8.273124694824219e-05 nb_pixel_total : 616 time to create 1 rle with old method : 0.0009624958038330078 length of segment : 22 time for calcul the mask position with numpy : 5.412101745605469e-05 nb_pixel_total : 640 time to create 1 rle with old method : 0.0009951591491699219 length of segment : 25 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -122.64141 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 0.0004286766052246094 nb_pixel_total : 7997 time to create 1 rle with old method : 0.023879528045654297 length of segment : 116 time for calcul the mask position with numpy : 0.00014925003051757812 nb_pixel_total : 226 time to create 1 rle with old method : 0.0006330013275146484 length of segment : 22 time for calcul the mask position with numpy : 0.0001728534698486328 nb_pixel_total : 1178 time to create 1 rle with old method : 0.004080772399902344 length of segment : 59 time for calcul the mask position with numpy : 9.322166442871094e-05 nb_pixel_total : 293 time to create 1 rle with old method : 0.0008869171142578125 length of segment : 23 time for calcul the mask position with numpy : 0.00011777877807617188 nb_pixel_total : 467 time to create 1 rle with old method : 0.0016438961029052734 length of segment : 25 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 245.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 117.13125 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 4.6253204345703125e-05 nb_pixel_total : 486 time to create 1 rle with old method : 0.0006883144378662109 length of segment : 19 time for calcul the mask position with numpy : 4.8160552978515625e-05 nb_pixel_total : 421 time to create 1 rle with old method : 0.0006120204925537109 length of segment : 17 time for calcul the mask position with numpy : 0.0005366802215576172 nb_pixel_total : 44548 time to create 1 rle with old method : 0.05193471908569336 length of segment : 286 Processing 1 images image shape: (400, 400, 3) min: 8.00000 max: 222.00000 molded_images shape: (1, 640, 640, 3) min: -88.00078 max: 88.86406 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 5.8650970458984375e-05 nb_pixel_total : 115 time to create 1 rle with old method : 0.0003197193145751953 length of segment : 20 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -110.33672 max: 150.63125 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 10 time for calcul the mask position with numpy : 7.033348083496094e-05 nb_pixel_total : 1211 time to create 1 rle with old method : 0.0015404224395751953 length of segment : 28 time for calcul the mask position with numpy : 4.863739013671875e-05 nb_pixel_total : 381 time to create 1 rle with old method : 0.0005543231964111328 length of segment : 19 time for calcul the mask position with numpy : 9.942054748535156e-05 nb_pixel_total : 3284 time to create 1 rle with old method : 0.00422358512878418 length of segment : 50 time for calcul the mask position with numpy : 5.078315734863281e-05 nb_pixel_total : 631 time to create 1 rle with old method : 0.0008733272552490234 length of segment : 33 time for calcul the mask position with numpy : 3.814697265625e-05 nb_pixel_total : 180 time to create 1 rle with old method : 0.00028824806213378906 length of segment : 12 time for calcul the mask position with numpy : 3.981590270996094e-05 nb_pixel_total : 187 time to create 1 rle with old method : 0.0003542900085449219 length of segment : 33 time for calcul the mask position with numpy : 6.651878356933594e-05 nb_pixel_total : 1652 time to create 1 rle with old method : 0.004418849945068359 length of segment : 50 time for calcul the mask position with numpy : 6.651878356933594e-05 nb_pixel_total : 762 time to create 1 rle with old method : 0.0010004043579101562 length of segment : 31 time for calcul the mask position with numpy : 0.0004458427429199219 nb_pixel_total : 16747 time to create 1 rle with old method : 0.019040822982788086 length of segment : 327 time for calcul the mask position with numpy : 0.0001289844512939453 nb_pixel_total : 3587 time to create 1 rle with old method : 0.004510164260864258 length of segment : 53 Processing 1 images image shape: (400, 400, 3) min: 2.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.77813 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 5.626678466796875e-05 nb_pixel_total : 1028 time to create 1 rle with old method : 0.0012288093566894531 length of segment : 45 time for calcul the mask position with numpy : 4.744529724121094e-05 nb_pixel_total : 592 time to create 1 rle with old method : 0.0009100437164306641 length of segment : 35 time for calcul the mask position with numpy : 7.128715515136719e-05 nb_pixel_total : 912 time to create 1 rle with old method : 0.0012726783752441406 length of segment : 40 time for calcul the mask position with numpy : 4.410743713378906e-05 nb_pixel_total : 680 time to create 1 rle with old method : 0.0010416507720947266 length of segment : 15 time for calcul the mask position with numpy : 5.0067901611328125e-05 nb_pixel_total : 681 time to create 1 rle with old method : 0.0010590553283691406 length of segment : 35 time for calcul the mask position with numpy : 3.457069396972656e-05 nb_pixel_total : 196 time to create 1 rle with old method : 0.00034499168395996094 length of segment : 11 time for calcul the mask position with numpy : 3.2901763916015625e-05 nb_pixel_total : 193 time to create 1 rle with old method : 0.00033855438232421875 length of segment : 12 time for calcul the mask position with numpy : 0.0002014636993408203 nb_pixel_total : 5981 time to create 1 rle with old method : 0.007384061813354492 length of segment : 149 time for calcul the mask position with numpy : 4.38690185546875e-05 nb_pixel_total : 144 time to create 1 rle with old method : 0.00022149085998535156 length of segment : 18 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -122.27813 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 7.152557373046875e-05 nb_pixel_total : 2238 time to create 1 rle with old method : 0.002698183059692383 length of segment : 55 time for calcul the mask position with numpy : 6.914138793945312e-05 nb_pixel_total : 695 time to create 1 rle with old method : 0.0011849403381347656 length of segment : 35 time for calcul the mask position with numpy : 5.221366882324219e-05 nb_pixel_total : 1263 time to create 1 rle with old method : 0.001611948013305664 length of segment : 39 time for calcul the mask position with numpy : 3.62396240234375e-05 nb_pixel_total : 116 time to create 1 rle with old method : 0.00019097328186035156 length of segment : 29 time for calcul the mask position with numpy : 0.0004858970642089844 nb_pixel_total : 6105 time to create 1 rle with old method : 0.0075016021728515625 length of segment : 305 time for calcul the mask position with numpy : 5.1021575927734375e-05 nb_pixel_total : 91 time to create 1 rle with old method : 0.0001785755157470703 length of segment : 9 time for calcul the mask position with numpy : 5.4836273193359375e-05 nb_pixel_total : 888 time to create 1 rle with old method : 0.0011010169982910156 length of segment : 39 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 121.17031 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 4.38690185546875e-05 nb_pixel_total : 462 time to create 1 rle with old method : 0.0006363391876220703 length of segment : 31 time for calcul the mask position with numpy : 3.719329833984375e-05 nb_pixel_total : 366 time to create 1 rle with old method : 0.0004870891571044922 length of segment : 34 time for calcul the mask position with numpy : 3.600120544433594e-05 nb_pixel_total : 376 time to create 1 rle with old method : 0.0005412101745605469 length of segment : 35 time for calcul the mask position with numpy : 5.2928924560546875e-05 nb_pixel_total : 963 time to create 1 rle with old method : 0.0012216567993164062 length of segment : 38 time for calcul the mask position with numpy : 5.793571472167969e-05 nb_pixel_total : 915 time to create 1 rle with old method : 0.0013039112091064453 length of segment : 52 time for calcul the mask position with numpy : 4.2438507080078125e-05 nb_pixel_total : 356 time to create 1 rle with old method : 0.0005068778991699219 length of segment : 42 time for calcul the mask position with numpy : 0.00022077560424804688 nb_pixel_total : 4337 time to create 1 rle with old method : 0.0050313472747802734 length of segment : 218 time for calcul the mask position with numpy : 5.364418029785156e-05 nb_pixel_total : 434 time to create 1 rle with old method : 0.0006060600280761719 length of segment : 43 time for calcul the mask position with numpy : 0.0002334117889404297 nb_pixel_total : 4377 time to create 1 rle with old method : 0.0056879520416259766 length of segment : 211 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 200.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 63.77422 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0011594295501708984 nb_pixel_total : 106814 time to create 1 rle with old method : 0.13966608047485352 length of segment : 284 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 145.69766 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 16 time for calcul the mask position with numpy : 0.0001347064971923828 nb_pixel_total : 3521 time to create 1 rle with old method : 0.005972146987915039 length of segment : 61 time for calcul the mask position with numpy : 6.127357482910156e-05 nb_pixel_total : 165 time to create 1 rle with old method : 0.00041484832763671875 length of segment : 11 time for calcul the mask position with numpy : 8.988380432128906e-05 nb_pixel_total : 1638 time to create 1 rle with old method : 0.0029294490814208984 length of segment : 73 time for calcul the mask position with numpy : 8.988380432128906e-05 nb_pixel_total : 732 time to create 1 rle with old method : 0.0013127326965332031 length of segment : 52 time for calcul the mask position with numpy : 6.961822509765625e-05 nb_pixel_total : 393 time to create 1 rle with old method : 0.0007624626159667969 length of segment : 31 time for calcul the mask position with numpy : 0.00015544891357421875 nb_pixel_total : 2412 time to create 1 rle with old method : 0.0042111873626708984 length of segment : 70 time for calcul the mask position with numpy : 7.605552673339844e-05 nb_pixel_total : 528 time to create 1 rle with old method : 0.0010073184967041016 length of segment : 29 time for calcul the mask position with numpy : 8.416175842285156e-05 nb_pixel_total : 647 time to create 1 rle with old method : 0.0012874603271484375 length of segment : 29 time for calcul the mask position with numpy : 0.00011277198791503906 nb_pixel_total : 765 time to create 1 rle with old method : 0.0014271736145019531 length of segment : 32 time for calcul the mask position with numpy : 6.413459777832031e-05 nb_pixel_total : 146 time to create 1 rle with old method : 0.0004165172576904297 length of segment : 17 time for calcul the mask position with numpy : 0.00013589859008789062 nb_pixel_total : 787 time to create 1 rle with old method : 0.0015950202941894531 length of segment : 34 time for calcul the mask position with numpy : 6.937980651855469e-05 nb_pixel_total : 210 time to create 1 rle with old method : 0.0004994869232177734 length of segment : 39 time for calcul the mask position with numpy : 7.224082946777344e-05 nb_pixel_total : 534 time to create 1 rle with old method : 0.0010161399841308594 length of segment : 28 time for calcul the mask position with numpy : 6.890296936035156e-05 nb_pixel_total : 528 time to create 1 rle with old method : 0.0010192394256591797 length of segment : 25 time for calcul the mask position with numpy : 7.677078247070312e-05 nb_pixel_total : 684 time to create 1 rle with old method : 0.001287221908569336 length of segment : 32 time for calcul the mask position with numpy : 8.344650268554688e-05 nb_pixel_total : 663 time to create 1 rle with old method : 0.0012569427490234375 length of segment : 30 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 148.26016 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 16 time for calcul the mask position with numpy : 7.271766662597656e-05 nb_pixel_total : 364 time to create 1 rle with old method : 0.0006806850433349609 length of segment : 27 time for calcul the mask position with numpy : 5.650520324707031e-05 nb_pixel_total : 466 time to create 1 rle with old method : 0.0008609294891357422 length of segment : 24 time for calcul the mask position with numpy : 0.00010514259338378906 nb_pixel_total : 377 time to create 1 rle with old method : 0.0008461475372314453 length of segment : 25 time for calcul the mask position with numpy : 6.842613220214844e-05 nb_pixel_total : 172 time to create 1 rle with old method : 0.00033473968505859375 length of segment : 21 time for calcul the mask position with numpy : 5.650520324707031e-05 nb_pixel_total : 298 time to create 1 rle with old method : 0.0005867481231689453 length of segment : 23 time for calcul the mask position with numpy : 5.91278076171875e-05 nb_pixel_total : 425 time to create 1 rle with old method : 0.000782012939453125 length of segment : 28 time for calcul the mask position with numpy : 5.1975250244140625e-05 nb_pixel_total : 85 time to create 1 rle with old method : 0.00021386146545410156 length of segment : 10 time for calcul the mask position with numpy : 6.079673767089844e-05 nb_pixel_total : 618 time to create 1 rle with old method : 0.0012192726135253906 length of segment : 22 time for calcul the mask position with numpy : 5.125999450683594e-05 nb_pixel_total : 151 time to create 1 rle with old method : 0.0003285408020019531 length of segment : 16 time for calcul the mask position with numpy : 4.76837158203125e-05 nb_pixel_total : 155 time to create 1 rle with old method : 0.00032520294189453125 length of segment : 17 time for calcul the mask position with numpy : 7.867813110351562e-05 nb_pixel_total : 396 time to create 1 rle with old method : 0.0007467269897460938 length of segment : 18 time for calcul the mask position with numpy : 4.673004150390625e-05 nb_pixel_total : 85 time to create 1 rle with old method : 0.0002231597900390625 length of segment : 8 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 211 time to create 1 rle with old method : 0.0003986358642578125 length of segment : 22 time for calcul the mask position with numpy : 6.031990051269531e-05 nb_pixel_total : 429 time to create 1 rle with old method : 0.0008177757263183594 length of segment : 24 time for calcul the mask position with numpy : 6.771087646484375e-05 nb_pixel_total : 638 time to create 1 rle with old method : 0.0011982917785644531 length of segment : 28 time for calcul the mask position with numpy : 6.0558319091796875e-05 nb_pixel_total : 439 time to create 1 rle with old method : 0.0008342266082763672 length of segment : 24 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.63125 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 5.555152893066406e-05 nb_pixel_total : 274 time to create 1 rle with old method : 0.0005445480346679688 length of segment : 16 time for calcul the mask position with numpy : 5.412101745605469e-05 nb_pixel_total : 586 time to create 1 rle with old method : 0.0010516643524169922 length of segment : 23 time for calcul the mask position with numpy : 0.00014472007751464844 nb_pixel_total : 21 time to create 1 rle with old method : 6.985664367675781e-05 length of segment : 12 time for calcul the mask position with numpy : 4.291534423828125e-05 nb_pixel_total : 254 time to create 1 rle with old method : 0.00046062469482421875 length of segment : 24 time for calcul the mask position with numpy : 6.103515625e-05 nb_pixel_total : 370 time to create 1 rle with old method : 0.000682830810546875 length of segment : 34 time for calcul the mask position with numpy : 8.273124694824219e-05 nb_pixel_total : 374 time to create 1 rle with old method : 0.0007262229919433594 length of segment : 20 time for calcul the mask position with numpy : 4.839897155761719e-05 nb_pixel_total : 101 time to create 1 rle with old method : 0.0002391338348388672 length of segment : 11 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 131.66250 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 5.221366882324219e-05 nb_pixel_total : 133 time to create 1 rle with old method : 0.00027632713317871094 length of segment : 55 Detection mask done ! Trying to reset tf kernel 4156292 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 467 tf kernel not reseted sub process len(results) : 215 len(list_Values) 215 None max_time_sub_proc : 3600 parent process len(results) : 0 len(list_Values) 215 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5756 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Catched exception ! Connect or reconnect ! Number saved : None batch 1 Loaded 215 chid ids of type : 4228 Number RLEs to save : 11400 save missing photos in datou_result : time spend for datou_step_exec : 24.00662398338318 time spend to save output : 0.8848254680633545 total time spend for step 1 : 24.891449451446533 step2:brightness Wed Feb 12 16:20:55 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1739373630_4155621_1337123742_c3a466a382624b326e53d7bc59be2ddc.jpg treat image : temp/1739373630_4155621_1337122899_94b05fe83a876e88366bb370c559c975.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 2 time used for this insertion : 0.013853073120117188 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 2 time used for this insertion : 0.012176275253295898 save missing photos in datou_result : time spend for datou_step_exec : 1.0608134269714355 time spend to save output : 0.031151294708251953 total time spend for step 2 : 1.0919647216796875 step3:blur_detection Wed Feb 12 16:20:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1739373630_4155621_1337123742_c3a466a382624b326e53d7bc59be2ddc.jpg resize: (1080, 1920) 1337123742 -7.775476818455898 treat image : temp/1739373630_4155621_1337122899_94b05fe83a876e88366bb370c559c975.jpg resize: (1080, 1920) 1337122899 -7.780035458387716 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 2 time used for this insertion : 0.013390779495239258 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 2 time used for this insertion : 0.011150836944580078 save missing photos in datou_result : time spend for datou_step_exec : 1.998687982559204 time spend to save output : 0.02950143814086914 total time spend for step 3 : 2.0281894207000732 step4:rle_unique_nms_with_priority Wed Feb 12 16:20:58 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 215 chid ids of type : 4228 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 107 nb_hashtags : 7 time to prepare the origin masks : 1.0908215045928955 time for calcul the mask position with numpy : 0.07377958297729492 nb_pixel_total : 1856083 time to create 1 rle with new method : 0.13788890838623047 time for calcul the mask position with numpy : 0.008690595626831055 nb_pixel_total : 170 time to create 1 rle with old method : 0.0002493858337402344 time for calcul the mask position with numpy : 0.009397029876708984 nb_pixel_total : 19411 time to create 1 rle with old method : 0.02173900604248047 time for calcul the mask position with numpy : 0.008780479431152344 nb_pixel_total : 4660 time to create 1 rle with old method : 0.005654811859130859 time for calcul the mask position with numpy : 0.008858203887939453 nb_pixel_total : 436 time to create 1 rle with old method : 0.0005414485931396484 time for calcul the mask position with numpy : 0.00904703140258789 nb_pixel_total : 830 time to create 1 rle with old method : 0.0010743141174316406 time for calcul the mask position with numpy : 0.00873708724975586 nb_pixel_total : 14 time to create 1 rle with old method : 2.8371810913085938e-05 time for calcul the mask position with numpy : 0.008959293365478516 nb_pixel_total : 102 time to create 1 rle with old method : 0.00013256072998046875 time for calcul the mask position with numpy : 0.008683443069458008 nb_pixel_total : 769 time to create 1 rle with old method : 0.0009453296661376953 time for calcul the mask position with numpy : 0.008640050888061523 nb_pixel_total : 16620 time to create 1 rle with old method : 0.019355058670043945 time for calcul the mask position with numpy : 0.008870601654052734 nb_pixel_total : 9823 time to create 1 rle with old method : 0.010774374008178711 time for calcul the mask position with numpy : 0.00853872299194336 nb_pixel_total : 3 time to create 1 rle with old method : 1.4066696166992188e-05 time for calcul the mask position with numpy : 0.00841212272644043 nb_pixel_total : 290 time to create 1 rle with old method : 0.0003609657287597656 time for calcul the mask position with numpy : 0.008403539657592773 nb_pixel_total : 439 time to create 1 rle with old method : 0.0005292892456054688 time for calcul the mask position with numpy : 0.008453130722045898 nb_pixel_total : 182 time to create 1 rle with old method : 0.0002624988555908203 time for calcul the mask position with numpy : 0.008477210998535156 nb_pixel_total : 37 time to create 1 rle with old method : 7.677078247070312e-05 time for calcul the mask position with numpy : 0.009173870086669922 nb_pixel_total : 38124 time to create 1 rle with old method : 0.04321169853210449 time for calcul the mask position with numpy : 0.008430719375610352 nb_pixel_total : 356 time to create 1 rle with old method : 0.0004241466522216797 time for calcul the mask position with numpy : 0.008379220962524414 nb_pixel_total : 203 time to create 1 rle with old method : 0.0002894401550292969 time for calcul the mask position with numpy : 0.008381843566894531 nb_pixel_total : 628 time to create 1 rle with old method : 0.0007255077362060547 time for calcul the mask position with numpy : 0.008341550827026367 nb_pixel_total : 304 time to create 1 rle with old method : 0.0003638267517089844 time for calcul the mask position with numpy : 0.008346796035766602 nb_pixel_total : 408 time to create 1 rle with old method : 0.00045943260192871094 time for calcul the mask position with numpy : 0.008381128311157227 nb_pixel_total : 1156 time to create 1 rle with old method : 0.0013294219970703125 time for calcul the mask position with numpy : 0.008805036544799805 nb_pixel_total : 4581 time to create 1 rle with old method : 0.005214214324951172 time for calcul the mask position with numpy : 0.008442878723144531 nb_pixel_total : 220 time to create 1 rle with old method : 0.00026679039001464844 time for calcul the mask position with numpy : 0.008436918258666992 nb_pixel_total : 680 time to create 1 rle with old method : 0.0011489391326904297 time for calcul the mask position with numpy : 0.009647846221923828 nb_pixel_total : 710 time to create 1 rle with old method : 0.0011761188507080078 time for calcul the mask position with numpy : 0.009618043899536133 nb_pixel_total : 97 time to create 1 rle with old method : 0.00018072128295898438 time for calcul the mask position with numpy : 0.01293635368347168 nb_pixel_total : 851 time to create 1 rle with old method : 0.0012333393096923828 time for calcul the mask position with numpy : 0.009130239486694336 nb_pixel_total : 769 time to create 1 rle with old method : 0.0008919239044189453 time for calcul the mask position with numpy : 0.008430004119873047 nb_pixel_total : 1196 time to create 1 rle with old method : 0.0014433860778808594 time for calcul the mask position with numpy : 0.00885319709777832 nb_pixel_total : 332 time to create 1 rle with old method : 0.0006589889526367188 time for calcul the mask position with numpy : 0.010671615600585938 nb_pixel_total : 462 time to create 1 rle with old method : 0.0007991790771484375 time for calcul the mask position with numpy : 0.010555744171142578 nb_pixel_total : 6216 time to create 1 rle with old method : 0.010325431823730469 time for calcul the mask position with numpy : 0.011320829391479492 nb_pixel_total : 8302 time to create 1 rle with old method : 0.014110803604125977 time for calcul the mask position with numpy : 0.011319160461425781 nb_pixel_total : 101 time to create 1 rle with old method : 0.0002334117889404297 time for calcul the mask position with numpy : 0.01128530502319336 nb_pixel_total : 643 time to create 1 rle with old method : 0.0011491775512695312 time for calcul the mask position with numpy : 0.011281013488769531 nb_pixel_total : 4 time to create 1 rle with old method : 4.4345855712890625e-05 time for calcul the mask position with numpy : 0.011327028274536133 nb_pixel_total : 472 time to create 1 rle with old method : 0.0008480548858642578 time for calcul the mask position with numpy : 0.011147499084472656 nb_pixel_total : 1298 time to create 1 rle with old method : 0.002317667007446289 time for calcul the mask position with numpy : 0.011634349822998047 nb_pixel_total : 19601 time to create 1 rle with old method : 0.034130096435546875 time for calcul the mask position with numpy : 0.011514425277709961 nb_pixel_total : 744 time to create 1 rle with old method : 0.0013155937194824219 time for calcul the mask position with numpy : 0.011275529861450195 nb_pixel_total : 1850 time to create 1 rle with old method : 0.0022847652435302734 time for calcul the mask position with numpy : 0.008502721786499023 nb_pixel_total : 9216 time to create 1 rle with old method : 0.010665655136108398 time for calcul the mask position with numpy : 0.0084991455078125 nb_pixel_total : 858 time to create 1 rle with old method : 0.0010530948638916016 time for calcul the mask position with numpy : 0.008505821228027344 nb_pixel_total : 335 time to create 1 rle with old method : 0.00046181678771972656 time for calcul the mask position with numpy : 0.00853419303894043 nb_pixel_total : 93 time to create 1 rle with old method : 0.00021338462829589844 time for calcul the mask position with numpy : 0.00860905647277832 nb_pixel_total : 4756 time to create 1 rle with old method : 0.0054476261138916016 time for calcul the mask position with numpy : 0.00852656364440918 nb_pixel_total : 72 time to create 1 rle with old method : 0.00015425682067871094 time for calcul the mask position with numpy : 0.008574962615966797 nb_pixel_total : 413 time to create 1 rle with old method : 0.0005004405975341797 time for calcul the mask position with numpy : 0.008428335189819336 nb_pixel_total : 1253 time to create 1 rle with old method : 0.001432657241821289 time for calcul the mask position with numpy : 0.008468151092529297 nb_pixel_total : 810 time to create 1 rle with old method : 0.0009560585021972656 time for calcul the mask position with numpy : 0.008460283279418945 nb_pixel_total : 3 time to create 1 rle with old method : 1.5974044799804688e-05 time for calcul the mask position with numpy : 0.008448362350463867 nb_pixel_total : 59 time to create 1 rle with old method : 7.557868957519531e-05 time for calcul the mask position with numpy : 0.008506059646606445 nb_pixel_total : 889 time to create 1 rle with old method : 0.001026153564453125 time for calcul the mask position with numpy : 0.008419036865234375 nb_pixel_total : 5076 time to create 1 rle with old method : 0.005728960037231445 time for calcul the mask position with numpy : 0.008476972579956055 nb_pixel_total : 778 time to create 1 rle with old method : 0.0009360313415527344 time for calcul the mask position with numpy : 0.008429288864135742 nb_pixel_total : 475 time to create 1 rle with old method : 0.0005934238433837891 time for calcul the mask position with numpy : 0.00836801528930664 nb_pixel_total : 87 time to create 1 rle with old method : 0.000217437744140625 time for calcul the mask position with numpy : 0.008311748504638672 nb_pixel_total : 3787 time to create 1 rle with old method : 0.004141569137573242 time for calcul the mask position with numpy : 0.008407354354858398 nb_pixel_total : 1384 time to create 1 rle with old method : 0.0016803741455078125 time for calcul the mask position with numpy : 0.008380413055419922 nb_pixel_total : 793 time to create 1 rle with old method : 0.0009257793426513672 time for calcul the mask position with numpy : 0.008378267288208008 nb_pixel_total : 1089 time to create 1 rle with old method : 0.0012841224670410156 time for calcul the mask position with numpy : 0.008387327194213867 nb_pixel_total : 39 time to create 1 rle with old method : 7.581710815429688e-05 time for calcul the mask position with numpy : 0.008367061614990234 nb_pixel_total : 393 time to create 1 rle with old method : 0.0004515647888183594 time for calcul the mask position with numpy : 0.008373260498046875 nb_pixel_total : 490 time to create 1 rle with old method : 0.0005841255187988281 time for calcul the mask position with numpy : 0.008408784866333008 nb_pixel_total : 245 time to create 1 rle with old method : 0.0002932548522949219 time for calcul the mask position with numpy : 0.008481740951538086 nb_pixel_total : 740 time to create 1 rle with old method : 0.0008676052093505859 time for calcul the mask position with numpy : 0.008493900299072266 nb_pixel_total : 13 time to create 1 rle with old method : 5.650520324707031e-05 time for calcul the mask position with numpy : 0.008664131164550781 nb_pixel_total : 188 time to create 1 rle with old method : 0.0002474784851074219 time for calcul the mask position with numpy : 0.008457183837890625 nb_pixel_total : 2875 time to create 1 rle with old method : 0.003231048583984375 time for calcul the mask position with numpy : 0.00866079330444336 nb_pixel_total : 365 time to create 1 rle with old method : 0.000415802001953125 time for calcul the mask position with numpy : 0.008593320846557617 nb_pixel_total : 1538 time to create 1 rle with old method : 0.0017964839935302734 time for calcul the mask position with numpy : 0.00857996940612793 nb_pixel_total : 1212 time to create 1 rle with old method : 0.0014214515686035156 time for calcul the mask position with numpy : 0.008519172668457031 nb_pixel_total : 1906 time to create 1 rle with old method : 0.002325296401977539 time for calcul the mask position with numpy : 0.008967161178588867 nb_pixel_total : 3452 time to create 1 rle with old method : 0.00424957275390625 time for calcul the mask position with numpy : 0.009273767471313477 nb_pixel_total : 128 time to create 1 rle with old method : 0.0002961158752441406 time for calcul the mask position with numpy : 0.008899927139282227 nb_pixel_total : 212 time to create 1 rle with old method : 0.0002739429473876953 time for calcul the mask position with numpy : 0.008563756942749023 nb_pixel_total : 183 time to create 1 rle with old method : 0.0002608299255371094 time for calcul the mask position with numpy : 0.008516073226928711 nb_pixel_total : 545 time to create 1 rle with old method : 0.0006113052368164062 time for calcul the mask position with numpy : 0.008470773696899414 nb_pixel_total : 85 time to create 1 rle with old method : 0.0001308917999267578 time for calcul the mask position with numpy : 0.008450508117675781 nb_pixel_total : 480 time to create 1 rle with old method : 0.0005905628204345703 time for calcul the mask position with numpy : 0.00851893424987793 nb_pixel_total : 780 time to create 1 rle with old method : 0.0009310245513916016 time for calcul the mask position with numpy : 0.008407354354858398 nb_pixel_total : 542 time to create 1 rle with old method : 0.0006647109985351562 time for calcul the mask position with numpy : 0.008428573608398438 nb_pixel_total : 2915 time to create 1 rle with old method : 0.003340482711791992 time for calcul the mask position with numpy : 0.008454322814941406 nb_pixel_total : 274 time to create 1 rle with old method : 0.0003330707550048828 time for calcul the mask position with numpy : 0.00863957405090332 nb_pixel_total : 12447 time to create 1 rle with old method : 0.01437687873840332 time for calcul the mask position with numpy : 0.008497476577758789 nb_pixel_total : 430 time to create 1 rle with old method : 0.0005292892456054688 time for calcul the mask position with numpy : 0.008517980575561523 nb_pixel_total : 285 time to create 1 rle with old method : 0.0003695487976074219 time for calcul the mask position with numpy : 0.008459329605102539 nb_pixel_total : 339 time to create 1 rle with old method : 0.0004260540008544922 time for calcul the mask position with numpy : 0.008514165878295898 nb_pixel_total : 1398 time to create 1 rle with old method : 0.0016448497772216797 time for calcul the mask position with numpy : 0.008624553680419922 nb_pixel_total : 179 time to create 1 rle with old method : 0.0002636909484863281 time for calcul the mask position with numpy : 0.008623600006103516 nb_pixel_total : 70 time to create 1 rle with old method : 0.0001671314239501953 time for calcul the mask position with numpy : 0.008840560913085938 nb_pixel_total : 384 time to create 1 rle with old method : 0.0004947185516357422 time for calcul the mask position with numpy : 0.008658409118652344 nb_pixel_total : 668 time to create 1 rle with old method : 0.0007898807525634766 time for calcul the mask position with numpy : 0.008729219436645508 nb_pixel_total : 40 time to create 1 rle with old method : 0.00012421607971191406 time for calcul the mask position with numpy : 0.010649442672729492 nb_pixel_total : 358 time to create 1 rle with old method : 0.000659942626953125 time for calcul the mask position with numpy : 0.01094961166381836 nb_pixel_total : 76 time to create 1 rle with old method : 0.00018596649169921875 time for calcul the mask position with numpy : 0.010898113250732422 nb_pixel_total : 227 time to create 1 rle with old method : 0.0004298686981201172 time for calcul the mask position with numpy : 0.01137089729309082 nb_pixel_total : 735 time to create 1 rle with old method : 0.0012531280517578125 time for calcul the mask position with numpy : 0.011000871658325195 nb_pixel_total : 378 time to create 1 rle with old method : 0.0008962154388427734 time for calcul the mask position with numpy : 0.011445283889770508 nb_pixel_total : 1260 time to create 1 rle with old method : 0.002301931381225586 time for calcul the mask position with numpy : 0.01105046272277832 nb_pixel_total : 460 time to create 1 rle with old method : 0.0008182525634765625 time for calcul the mask position with numpy : 0.011429309844970703 nb_pixel_total : 542 time to create 1 rle with old method : 0.001055002212524414 time for calcul the mask position with numpy : 0.011153936386108398 nb_pixel_total : 425 time to create 1 rle with old method : 0.0008282661437988281 time for calcul the mask position with numpy : 0.011534452438354492 nb_pixel_total : 595 time to create 1 rle with old method : 0.0010941028594970703 time for calcul the mask position with numpy : 0.011331319808959961 nb_pixel_total : 102 time to create 1 rle with old method : 0.0002512931823730469 time for calcul the mask position with numpy : 0.010421276092529297 nb_pixel_total : 199 time to create 1 rle with old method : 0.00025272369384765625 create new chi : 1.4871609210968018 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.006615638732910156 batch 1 Loaded 108 chid ids of type : 4230 Number RLEs to save : 12359 TO DO : save crop sub photo not yet done ! save time : 0.7558684349060059 nb_obj : 105 nb_hashtags : 7 time to prepare the origin masks : 2.2390005588531494 time for calcul the mask position with numpy : 0.04364180564880371 nb_pixel_total : 1819477 time to create 1 rle with new method : 0.21446013450622559 time for calcul the mask position with numpy : 0.0072917938232421875 nb_pixel_total : 3624 time to create 1 rle with old method : 0.0048122406005859375 time for calcul the mask position with numpy : 0.016176700592041016 nb_pixel_total : 292 time to create 1 rle with old method : 0.0004401206970214844 time for calcul the mask position with numpy : 0.014875650405883789 nb_pixel_total : 44548 time to create 1 rle with old method : 0.05329084396362305 time for calcul the mask position with numpy : 0.010004281997680664 nb_pixel_total : 847 time to create 1 rle with old method : 0.0010142326354980469 time for calcul the mask position with numpy : 0.009511470794677734 nb_pixel_total : 42 time to create 1 rle with old method : 6.890296936035156e-05 time for calcul the mask position with numpy : 0.006754636764526367 nb_pixel_total : 70 time to create 1 rle with old method : 0.00011301040649414062 time for calcul the mask position with numpy : 0.006499528884887695 nb_pixel_total : 43 time to create 1 rle with old method : 7.915496826171875e-05 time for calcul the mask position with numpy : 0.00649571418762207 nb_pixel_total : 222 time to create 1 rle with old method : 0.00032019615173339844 time for calcul the mask position with numpy : 0.006598472595214844 nb_pixel_total : 267 time to create 1 rle with old method : 0.0004904270172119141 time for calcul the mask position with numpy : 0.006663799285888672 nb_pixel_total : 127 time to create 1 rle with old method : 0.0002727508544921875 time for calcul the mask position with numpy : 0.0072269439697265625 nb_pixel_total : 2527 time to create 1 rle with old method : 0.0042018890380859375 time for calcul the mask position with numpy : 0.007357120513916016 nb_pixel_total : 16 time to create 1 rle with old method : 9.298324584960938e-05 time for calcul the mask position with numpy : 0.006816864013671875 nb_pixel_total : 8 time to create 1 rle with old method : 3.62396240234375e-05 time for calcul the mask position with numpy : 0.006482601165771484 nb_pixel_total : 1178 time to create 1 rle with old method : 0.0013549327850341797 time for calcul the mask position with numpy : 0.00788736343383789 nb_pixel_total : 226 time to create 1 rle with old method : 0.0003094673156738281 time for calcul the mask position with numpy : 0.007085084915161133 nb_pixel_total : 5 time to create 1 rle with old method : 2.2411346435546875e-05 time for calcul the mask position with numpy : 0.006482839584350586 nb_pixel_total : 822 time to create 1 rle with old method : 0.0009734630584716797 time for calcul the mask position with numpy : 0.0070362091064453125 nb_pixel_total : 640 time to create 1 rle with old method : 0.0007808208465576172 time for calcul the mask position with numpy : 0.007018089294433594 nb_pixel_total : 467 time to create 1 rle with old method : 0.0006527900695800781 time for calcul the mask position with numpy : 0.006822824478149414 nb_pixel_total : 1394 time to create 1 rle with old method : 0.0016326904296875 time for calcul the mask position with numpy : 0.006411314010620117 nb_pixel_total : 446 time to create 1 rle with old method : 0.0005199909210205078 time for calcul the mask position with numpy : 0.006541728973388672 nb_pixel_total : 616 time to create 1 rle with old method : 0.0007545948028564453 time for calcul the mask position with numpy : 0.006364583969116211 nb_pixel_total : 1262 time to create 1 rle with old method : 0.00151824951171875 time for calcul the mask position with numpy : 0.006755352020263672 nb_pixel_total : 333 time to create 1 rle with old method : 0.00039839744567871094 time for calcul the mask position with numpy : 0.006930112838745117 nb_pixel_total : 5381 time to create 1 rle with old method : 0.0061740875244140625 time for calcul the mask position with numpy : 0.006878852844238281 nb_pixel_total : 309 time to create 1 rle with old method : 0.0003769397735595703 time for calcul the mask position with numpy : 0.006609916687011719 nb_pixel_total : 7997 time to create 1 rle with old method : 0.008984088897705078 time for calcul the mask position with numpy : 0.0069582462310791016 nb_pixel_total : 486 time to create 1 rle with old method : 0.0006463527679443359 time for calcul the mask position with numpy : 0.006246805191040039 nb_pixel_total : 187 time to create 1 rle with old method : 0.0002741813659667969 time for calcul the mask position with numpy : 0.006476163864135742 nb_pixel_total : 5 time to create 1 rle with old method : 4.506111145019531e-05 time for calcul the mask position with numpy : 0.006842851638793945 nb_pixel_total : 196 time to create 1 rle with old method : 0.0002510547637939453 time for calcul the mask position with numpy : 0.0064389705657958984 nb_pixel_total : 6105 time to create 1 rle with old method : 0.007082939147949219 time for calcul the mask position with numpy : 0.006349325180053711 nb_pixel_total : 2238 time to create 1 rle with old method : 0.002652406692504883 time for calcul the mask position with numpy : 0.0067293643951416016 nb_pixel_total : 888 time to create 1 rle with old method : 0.001070261001586914 time for calcul the mask position with numpy : 0.006574153900146484 nb_pixel_total : 16747 time to create 1 rle with old method : 0.018713951110839844 time for calcul the mask position with numpy : 0.006827592849731445 nb_pixel_total : 5418 time to create 1 rle with old method : 0.006059408187866211 time for calcul the mask position with numpy : 0.006624460220336914 nb_pixel_total : 680 time to create 1 rle with old method : 0.0007817745208740234 time for calcul the mask position with numpy : 0.00640416145324707 nb_pixel_total : 1263 time to create 1 rle with old method : 0.00157928466796875 time for calcul the mask position with numpy : 0.007047891616821289 nb_pixel_total : 762 time to create 1 rle with old method : 0.0009303092956542969 time for calcul the mask position with numpy : 0.006756305694580078 nb_pixel_total : 116 time to create 1 rle with old method : 0.00016164779663085938 time for calcul the mask position with numpy : 0.006926774978637695 nb_pixel_total : 115 time to create 1 rle with old method : 0.0001621246337890625 time for calcul the mask position with numpy : 0.0065152645111083984 nb_pixel_total : 963 time to create 1 rle with old method : 0.0011317729949951172 time for calcul the mask position with numpy : 0.006372213363647461 nb_pixel_total : 912 time to create 1 rle with old method : 0.0010933876037597656 time for calcul the mask position with numpy : 0.006432294845581055 nb_pixel_total : 4337 time to create 1 rle with old method : 0.004971504211425781 time for calcul the mask position with numpy : 0.006432294845581055 nb_pixel_total : 233 time to create 1 rle with old method : 0.00036597251892089844 time for calcul the mask position with numpy : 0.0071468353271484375 nb_pixel_total : 86 time to create 1 rle with old method : 0.00014400482177734375 time for calcul the mask position with numpy : 0.00649571418762207 nb_pixel_total : 356 time to create 1 rle with old method : 0.0004425048828125 time for calcul the mask position with numpy : 0.0065765380859375 nb_pixel_total : 631 time to create 1 rle with old method : 0.000728607177734375 time for calcul the mask position with numpy : 0.00706934928894043 nb_pixel_total : 1028 time to create 1 rle with old method : 0.0021500587463378906 time for calcul the mask position with numpy : 0.008445501327514648 nb_pixel_total : 91 time to create 1 rle with old method : 0.00018095970153808594 time for calcul the mask position with numpy : 0.0073812007904052734 nb_pixel_total : 381 time to create 1 rle with old method : 0.0006747245788574219 time for calcul the mask position with numpy : 0.010159969329833984 nb_pixel_total : 592 time to create 1 rle with old method : 0.0009970664978027344 time for calcul the mask position with numpy : 0.016185283660888672 nb_pixel_total : 102 time to create 1 rle with old method : 0.0002570152282714844 time for calcul the mask position with numpy : 0.01175999641418457 nb_pixel_total : 462 time to create 1 rle with old method : 0.0005636215209960938 time for calcul the mask position with numpy : 0.010516166687011719 nb_pixel_total : 144 time to create 1 rle with old method : 0.0001881122589111328 time for calcul the mask position with numpy : 0.010266304016113281 nb_pixel_total : 915 time to create 1 rle with old method : 0.0012450218200683594 time for calcul the mask position with numpy : 0.012851953506469727 nb_pixel_total : 180 time to create 1 rle with old method : 0.00024580955505371094 time for calcul the mask position with numpy : 0.013476133346557617 nb_pixel_total : 3284 time to create 1 rle with old method : 0.004439353942871094 time for calcul the mask position with numpy : 0.013249397277832031 nb_pixel_total : 408 time to create 1 rle with old method : 0.0007312297821044922 time for calcul the mask position with numpy : 0.01743006706237793 nb_pixel_total : 366 time to create 1 rle with old method : 0.00067138671875 time for calcul the mask position with numpy : 0.015127182006835938 nb_pixel_total : 28 time to create 1 rle with old method : 0.00011372566223144531 time for calcul the mask position with numpy : 0.017200708389282227 nb_pixel_total : 1652 time to create 1 rle with old method : 0.0034079551696777344 time for calcul the mask position with numpy : 0.00951695442199707 nb_pixel_total : 695 time to create 1 rle with old method : 0.0012998580932617188 time for calcul the mask position with numpy : 0.007670164108276367 nb_pixel_total : 1211 time to create 1 rle with old method : 0.002210855484008789 time for calcul the mask position with numpy : 0.009237051010131836 nb_pixel_total : 106814 time to create 1 rle with old method : 0.15626835823059082 time for calcul the mask position with numpy : 0.009447813034057617 nb_pixel_total : 165 time to create 1 rle with old method : 0.0003495216369628906 time for calcul the mask position with numpy : 0.010123014450073242 nb_pixel_total : 3521 time to create 1 rle with old method : 0.006096839904785156 time for calcul the mask position with numpy : 0.006857633590698242 nb_pixel_total : 1624 time to create 1 rle with old method : 0.002552032470703125 time for calcul the mask position with numpy : 0.006984710693359375 nb_pixel_total : 133 time to create 1 rle with old method : 0.00026297569274902344 time for calcul the mask position with numpy : 0.007080078125 nb_pixel_total : 155 time to create 1 rle with old method : 0.00026798248291015625 time for calcul the mask position with numpy : 0.007441520690917969 nb_pixel_total : 191 time to create 1 rle with old method : 0.00024580955505371094 time for calcul the mask position with numpy : 0.006447553634643555 nb_pixel_total : 211 time to create 1 rle with old method : 0.0002665519714355469 time for calcul the mask position with numpy : 0.00647735595703125 nb_pixel_total : 85 time to create 1 rle with old method : 0.00011467933654785156 time for calcul the mask position with numpy : 0.006491899490356445 nb_pixel_total : 618 time to create 1 rle with old method : 0.0007343292236328125 time for calcul the mask position with numpy : 0.006300926208496094 nb_pixel_total : 274 time to create 1 rle with old method : 0.0003275871276855469 time for calcul the mask position with numpy : 0.006337404251098633 nb_pixel_total : 528 time to create 1 rle with old method : 0.0006005764007568359 time for calcul the mask position with numpy : 0.0064008235931396484 nb_pixel_total : 787 time to create 1 rle with old method : 0.0009243488311767578 time for calcul the mask position with numpy : 0.006436824798583984 nb_pixel_total : 528 time to create 1 rle with old method : 0.0006344318389892578 time for calcul the mask position with numpy : 0.006499290466308594 nb_pixel_total : 16 time to create 1 rle with old method : 5.3882598876953125e-05 time for calcul the mask position with numpy : 0.006199359893798828 nb_pixel_total : 146 time to create 1 rle with old method : 0.00018548965454101562 time for calcul the mask position with numpy : 0.006478548049926758 nb_pixel_total : 429 time to create 1 rle with old method : 0.0005006790161132812 time for calcul the mask position with numpy : 0.006426334381103516 nb_pixel_total : 2412 time to create 1 rle with old method : 0.0026884078979492188 time for calcul the mask position with numpy : 0.006451845169067383 nb_pixel_total : 210 time to create 1 rle with old method : 0.0002791881561279297 time for calcul the mask position with numpy : 0.006589651107788086 nb_pixel_total : 12 time to create 1 rle with old method : 5.054473876953125e-05 time for calcul the mask position with numpy : 0.0065500736236572266 nb_pixel_total : 298 time to create 1 rle with old method : 0.0003681182861328125 time for calcul the mask position with numpy : 0.0064694881439208984 nb_pixel_total : 425 time to create 1 rle with old method : 0.0005023479461669922 time for calcul the mask position with numpy : 0.006464719772338867 nb_pixel_total : 172 time to create 1 rle with old method : 0.0002231597900390625 time for calcul the mask position with numpy : 0.006276845932006836 nb_pixel_total : 393 time to create 1 rle with old method : 0.0004591941833496094 time for calcul the mask position with numpy : 0.006476640701293945 nb_pixel_total : 101 time to create 1 rle with old method : 0.0001697540283203125 time for calcul the mask position with numpy : 0.006408214569091797 nb_pixel_total : 396 time to create 1 rle with old method : 0.00046563148498535156 time for calcul the mask position with numpy : 0.007230043411254883 nb_pixel_total : 46 time to create 1 rle with old method : 0.00012612342834472656 time for calcul the mask position with numpy : 0.007685661315917969 nb_pixel_total : 647 time to create 1 rle with old method : 0.0009648799896240234 time for calcul the mask position with numpy : 0.006535530090332031 nb_pixel_total : 374 time to create 1 rle with old method : 0.0004639625549316406 time for calcul the mask position with numpy : 0.006395816802978516 nb_pixel_total : 765 time to create 1 rle with old method : 0.0009112358093261719 time for calcul the mask position with numpy : 0.0063855648040771484 nb_pixel_total : 2 time to create 1 rle with old method : 5.364418029785156e-05 time for calcul the mask position with numpy : 0.006400585174560547 nb_pixel_total : 364 time to create 1 rle with old method : 0.0004918575286865234 time for calcul the mask position with numpy : 0.006331443786621094 nb_pixel_total : 85 time to create 1 rle with old method : 0.0001461505889892578 time for calcul the mask position with numpy : 0.006429195404052734 nb_pixel_total : 21 time to create 1 rle with old method : 4.8160552978515625e-05 time for calcul the mask position with numpy : 0.006380319595336914 nb_pixel_total : 253 time to create 1 rle with old method : 0.0003108978271484375 time for calcul the mask position with numpy : 0.008459806442260742 nb_pixel_total : 732 time to create 1 rle with old method : 0.0008752346038818359 time for calcul the mask position with numpy : 0.007404804229736328 nb_pixel_total : 377 time to create 1 rle with old method : 0.0004601478576660156 time for calcul the mask position with numpy : 0.006293058395385742 nb_pixel_total : 466 time to create 1 rle with old method : 0.0005552768707275391 time for calcul the mask position with numpy : 0.006386518478393555 nb_pixel_total : 586 time to create 1 rle with old method : 0.0006823539733886719 time for calcul the mask position with numpy : 0.006481647491455078 nb_pixel_total : 151 time to create 1 rle with old method : 0.00019359588623046875 time for calcul the mask position with numpy : 0.0066411495208740234 nb_pixel_total : 638 time to create 1 rle with old method : 0.0007224082946777344 create new chi : 1.42759370803833 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.0013904571533203125 batch 1 Loaded 106 chid ids of type : 4230 Number RLEs to save : 10131 TO DO : save crop sub photo not yet done ! save time : 0.8006429672241211 map_output_result : {1337123742: (0.0, 'Should be the crop_list due to order', 0.0), 1337122899: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1337123742, 1337122899] Looping around the photos to save general results len do output : 2 /1337123742.Didn't retrieve data . /1337122899.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2579746') ('4234', '20479443', '1337123742', None, None, None, None, None, '2579746') ('4234', None, None, None, None, None, None, None, '2579746') ('4234', '20479443', '1337122899', None, None, None, None, None, '2579746') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.012108325958251953 save_final save missing photos in datou_result : time spend for datou_step_exec : 8.052573204040527 time spend to save output : 0.012487649917602539 total time spend for step 4 : 8.06506085395813 step5:crop_condition Wed Feb 12 16:21:06 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4230 Loading chi in step crop for list_pids : 2 ! batch 1 Loaded 214 chid ids of type : 4230 begin to crop the class : papier param for this class : {'min_score': 0.6} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 67 About to insert : list_path_to_insert length 67 new photo from crops ! About to upload 67 photos upload in portfolio : 4869462 init cache_photo without model_param we have 67 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739373668_4155621 we have uploaded 67 photos in the portfolio 4869462 time of upload the photos Elapsed time : 15.411613702774048 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.6} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 40 About to insert : list_path_to_insert length 40 new photo from crops ! About to upload 40 photos upload in portfolio : 4869462 init cache_photo without model_param we have 40 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739373684_4155621 we have uploaded 40 photos in the portfolio 4869462 time of upload the photos Elapsed time : 9.762008666992188 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.6} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.6} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! About to upload 7 photos upload in portfolio : 4869462 init cache_photo without model_param we have 7 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739373694_4155621 we have uploaded 7 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.2484560012817383 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.6} filtre for class : autre hashtag_id of this class : 494826614 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 29 About to insert : list_path_to_insert length 29 new photo from crops ! About to upload 29 photos upload in portfolio : 4869462 init cache_photo without model_param we have 29 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739373697_4155621 we have uploaded 29 photos in the portfolio 4869462 time of upload the photos Elapsed time : 8.234247922897339 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.6} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.6} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1337123742, 1337122899] Looping around the photos to save general results len do output : 143 /1337131789Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131791Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131793Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131795Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131797Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131799Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131801Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131803Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131805Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131807Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131809Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131811Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131813Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131815Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131817Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131819Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131821Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131823Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131825Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131827Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131829Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131831Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131833Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131835Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131837Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131839Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131841Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131843Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131846Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131850Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131854Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131858Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131862Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131866Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131868Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131870Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131872Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131874Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131876Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131878Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131881Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131883Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131885Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131888Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131892Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131895Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131897Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131899Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131901Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131903Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131905Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131907Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131909Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131911Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131913Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131915Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131917Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131919Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131921Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131923Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131925Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131927Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131929Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131931Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131933Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131935Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337131937Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132141Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132143Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132145Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132147Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132149Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132151Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132153Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132155Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132157Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132159Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132161Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132163Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132165Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132167Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132169Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132171Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132173Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132175Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132177Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132179Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132181Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132183Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132185Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132187Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132189Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132191Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132193Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132195Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132197Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132199Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132200Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132201Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132202Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132203Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132204Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132205Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132206Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132207Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132208Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132209Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132211Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132212Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132213Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132214Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132215Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132216Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132217Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132303Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132304Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132305Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132306Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132307Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132308Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132309Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132310Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132311Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132312Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132313Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132314Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132315Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132316Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132317Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132318Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132319Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132320Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132321Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132322Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132323Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132324Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132325Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132326Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132327Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132328Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132329Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132330Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1337132331Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2579746') ('4234', '20479443', '1337123742', None, None, None, None, None, '2579746') ('4234', None, None, None, None, None, None, None, '2579746') ('4234', '20479443', '1337122899', None, None, None, None, None, '2579746') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 431 time used for this insertion : 0.033873558044433594 save_final save missing photos in datou_result : time spend for datou_step_exec : 38.6065890789032 time spend to save output : 0.044656991958618164 total time spend for step 5 : 38.651246070861816 step6:thcl Wed Feb 12 16:21:45 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! we are using the classfication for only one thcl 3237 time to import caffe and check if the image exist : 0.006582975387573242 time to convert the images to numpy array : 0.011460304260253906 time to import caffe and check if the image exist : 0.006514549255371094 time to convert the images to numpy array : 0.04033970832824707 time to import caffe and check if the image exist : 0.013128519058227539 time to convert the images to numpy array : 0.03652501106262207 time to import caffe and check if the image exist : 0.013824701309204102 time to convert the images to numpy array : 0.036763668060302734 time to import caffe and check if the image exist : 0.018987178802490234 time to convert the images to numpy array : 0.03201127052307129 time to import caffe and check if the image exist : 0.01378774642944336 time to convert the images to numpy array : 0.038335561752319336 time to import caffe and check if the image exist : 0.011608123779296875 time to convert the images to numpy array : 0.041210174560546875 time to import caffe and check if the image exist : 0.012754678726196289 time to convert the images to numpy array : 0.041991472244262695 time to import caffe and check if the image exist : 0.02173638343811035 time to convert the images to numpy array : 0.033164262771606445 time to import caffe and check if the image exist : 0.012178659439086914 time to convert the images to numpy array : 0.04330635070800781 total time to convert the images to numpy array : 0.5052082538604736 list photo_ids error: [] list photo_ids correct : [1337132324, 1337132325, 1337132326, 1337132327, 1337132328, 1337132329, 1337132330, 1337132331, 1337131789, 1337131791, 1337131793, 1337131795, 1337131797, 1337131799, 1337131801, 1337131803, 1337131805, 1337131807, 1337131809, 1337131811, 1337131813, 1337131815, 1337131817, 1337131895, 1337131897, 1337131899, 1337131901, 1337131903, 1337131905, 1337131907, 1337131909, 1337131911, 1337131913, 1337131915, 1337131917, 1337131919, 1337131921, 1337131923, 1337131854, 1337131858, 1337131862, 1337131866, 1337131868, 1337131870, 1337131872, 1337131874, 1337131876, 1337131878, 1337131881, 1337131883, 1337131885, 1337131888, 1337131892, 1337132309, 1337132310, 1337132311, 1337132312, 1337132313, 1337132314, 1337132315, 1337132316, 1337132317, 1337132318, 1337132319, 1337132320, 1337132321, 1337132322, 1337132323, 1337132187, 1337132189, 1337132191, 1337132193, 1337132195, 1337132197, 1337132199, 1337132200, 1337132201, 1337132202, 1337132203, 1337132204, 1337132205, 1337132206, 1337132207, 1337131925, 1337131927, 1337131929, 1337131931, 1337131933, 1337131935, 1337131937, 1337132141, 1337132143, 1337132145, 1337132147, 1337132149, 1337132151, 1337132153, 1337132155, 1337132157, 1337132159, 1337132161, 1337132163, 1337132165, 1337132167, 1337132169, 1337132171, 1337132173, 1337132175, 1337132177, 1337132179, 1337132181, 1337132183, 1337132185, 1337132208, 1337132209, 1337132211, 1337132212, 1337132213, 1337132214, 1337132215, 1337132216, 1337132217, 1337132303, 1337132304, 1337132305, 1337132306, 1337132307, 1337132308, 1337131819, 1337131821, 1337131823, 1337131825, 1337131827, 1337131829, 1337131831, 1337131833, 1337131835, 1337131837, 1337131839, 1337131841, 1337131843, 1337131846, 1337131850] number of photos to traite : 143 try to delete the photos incorrect in DB tagging for thcl : 3237 To do loadFromThcl(), then load ParamDescType : thcl3237 thcls : [{'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'}] thcl {'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'} Update svm_hashtag_type_desc : 5561 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) To loadFromThcl() : net_5561 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 8365 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_rubbia_refus_2500 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_rubbia_refus_2500 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_rubbia_refus_2500 /data/models_weight/learn_rubbia_refus_2500/caffemodel size_local : 94358479 size in s3 : 94358479 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:16 caffemodel already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt size_local : 32544 size in s3 : 32544 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:15 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:09:52 mean.npy already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/synset_words.txt size_local : 334 size in s3 : 334 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:10:06 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt caffemodel_filename : /data/models_weight/learn_rubbia_refus_2500/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 8146 max_wait_temp : 1 max_wait : 0 dict_keys(['pool5', 'prob']) time used to do the prepocess of the images : 0.7529964447021484 time used to do the prediction : 0.745229959487915 save descriptor for thcl : 3237 time to traite the descriptors : 0.9356551170349121 Catched exception ! Connect or reconnect ! storage_type for insertDescriptorsMulti : 3 To insert : 1337132324 To insert : 1337132325 To insert : 1337132326 To insert : 1337132327 To insert : 1337132328 To insert : 1337132329 To insert : 1337132330 To insert : 1337132331 To insert : 1337131789 To insert : 1337131791 To insert : 1337131793 To insert : 1337131795 To insert : 1337131797 To insert : 1337131799 To insert : 1337131801 To insert : 1337131803 To insert : 1337131805 To insert : 1337131807 To insert : 1337131809 To insert : 1337131811 To insert : 1337131813 To insert : 1337131815 To insert : 1337131817 To insert : 1337131895 To insert : 1337131897 To insert : 1337131899 To insert : 1337131901 To insert : 1337131903 To insert : 1337131905 To insert : 1337131907 To insert : 1337131909 To insert : 1337131911 To insert : 1337131913 To insert : 1337131915 To insert : 1337131917 To insert : 1337131919 To insert : 1337131921 To insert : 1337131923 To insert : 1337131854 To insert : 1337131858 To insert : 1337131862 To insert : 1337131866 To insert : 1337131868 To insert : 1337131870 To insert : 1337131872 To insert : 1337131874 To insert : 1337131876 To insert : 1337131878 To insert : 1337131881 To insert : 1337131883 To insert : 1337131885 To insert : 1337131888 To insert : 1337131892 To insert : 1337132309 To insert : 1337132310 To insert : 1337132311 To insert : 1337132312 To insert : 1337132313 To insert : 1337132314 To insert : 1337132315 To insert : 1337132316 To insert : 1337132317 To insert : 1337132318 To insert : 1337132319 To insert : 1337132320 To insert : 1337132321 To insert : 1337132322 To insert : 1337132323 To insert : 1337132187 To insert : 1337132189 To insert : 1337132191 To insert : 1337132193 To insert : 1337132195 To insert : 1337132197 To insert : 1337132199 To insert : 1337132200 To insert : 1337132201 To insert : 1337132202 To insert : 1337132203 To insert : 1337132204 To insert : 1337132205 To insert : 1337132206 To insert : 1337132207 To insert : 1337131925 To insert : 1337131927 To insert : 1337131929 To insert : 1337131931 To insert : 1337131933 To insert : 1337131935 To insert : 1337131937 To insert : 1337132141 To insert : 1337132143 To insert : 1337132145 To insert : 1337132147 To insert : 1337132149 To insert : 1337132151 To insert : 1337132153 To insert : 1337132155 To insert : 1337132157 To insert : 1337132159 To insert : 1337132161 To insert : 1337132163 To insert : 1337132165 To insert : 1337132167 To insert : 1337132169 To insert : 1337132171 To insert : 1337132173 To insert : 1337132175 To insert : 1337132177 To insert : 1337132179 To insert : 1337132181 To insert : 1337132183 To insert : 1337132185 To insert : 1337132208 To insert : 1337132209 To insert : 1337132211 To insert : 1337132212 To insert : 1337132213 To insert : 1337132214 To insert : 1337132215 To insert : 1337132216 To insert : 1337132217 To insert : 1337132303 To insert : 1337132304 To insert : 1337132305 To insert : 1337132306 To insert : 1337132307 To insert : 1337132308 To insert : 1337131819 To insert : 1337131821 To insert : 1337131823 To insert : 1337131825 To insert : 1337131827 To insert : 1337131829 To insert : 1337131831 To insert : 1337131833 To insert : 1337131835 To insert : 1337131837 To insert : 1337131839 To insert : 1337131841 To insert : 1337131843 To insert : 1337131846 To insert : 1337131850 time to insert the descriptors : 31.93060827255249 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos SAVE THCL : begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 232 time used for this insertion : 0.023253202438354492 save missing photos in datou_result : time spend for datou_step_exec : 39.268038272857666 time spend to save output : 0.8230879306793213 total time spend for step 6 : 40.09112620353699 step7:ventilate_hashtags_in_portfolio Wed Feb 12 16:22:25 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 20479443 get user id for portfolio 20479443 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20479443 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','flou','PEHD','PET_fonce','pet_clair','papier','pet_fonce','Carton','Tetrapak','PET_clair','Papier','autre','carton','environnement','refus','metal','Film_plastique','pehd')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20479443 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','flou','PEHD','PET_fonce','pet_clair','papier','pet_fonce','Carton','Tetrapak','PET_clair','Papier','autre','carton','environnement','refus','metal','Film_plastique','pehd')) AND mptpi.`min_score`=0.6 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20479443 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','flou','PEHD','PET_fonce','pet_clair','papier','pet_fonce','Carton','Tetrapak','PET_clair','Papier','autre','carton','environnement','refus','metal','Film_plastique','pehd')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20479443 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','flou','PEHD','PET_fonce','pet_clair','papier','pet_fonce','Carton','Tetrapak','PET_clair','Papier','autre','carton','environnement','refus','metal','Film_plastique','pehd')) AND mptpi.`min_score`=0.6 To do lien utilise dans velours : https://www.fotonower.com/velours/20480175,20480176,20480192,20480181,20480184,20480185,20480187,20480183,20480186,20480188,20480189,20480190,20480191?tags=mal_croppe,flou,pehd,pet_fonce,pet_clair,papier,carton,tetrapak,autre,environnement,refus,metal,film_plastique&datou_id_consolidate=4235&port_consolidate=20479443 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1337123742, 1337122899] Looping around the photos to save general results len do output : 1 /20479443. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2579746') ('4234', '20479443', '1337123742', None, None, None, None, None, '2579746') ('4234', None, None, None, None, None, None, None, '2579746') ('4234', '20479443', '1337122899', None, None, None, None, None, '2579746') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.015366315841674805 save_final save missing photos in datou_result : time spend for datou_step_exec : 5.308911561965942 time spend to save output : 0.015595197677612305 total time spend for step 7 : 5.324506759643555 step8:final Wed Feb 12 16:22:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1337123742: ('0.22406120432686213',), 1337122899: ('0.22406120432686213',)} new output for save of step final : {1337123742: ('0.22406120432686213',), 1337122899: ('0.22406120432686213',)} [1337123742, 1337122899] Looping around the photos to save general results len do output : 2 /1337123742.Didn't retrieve data . /1337122899.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2579746') ('4234', '20479443', '1337123742', None, None, None, None, None, '2579746') ('4234', None, None, None, None, None, None, None, '2579746') ('4234', '20479443', '1337122899', None, None, None, None, None, '2579746') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.012464761734008789 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.15325617790222168 time spend to save output : 0.012662887573242188 total time spend for step 8 : 0.16591906547546387 step9:velours_tree Wed Feb 12 16:22:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure list_portfolios : 20480175,20480176,20480192,20480181,20480184,20480185,20480187,20480183,20480186,20480188,20480189,20480190,20480191 photo desc type : 5561 - Retrieving photos to tag... query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20480175 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20480176 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20480192 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20480181 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20480184 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20480185 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20480187 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20480183 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20480186 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20480188 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20480189 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20480190 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20480191 ORDER BY ph.size desc - Loading descriptors... Size : 2048 len(descriptors) : 143 Compute structured hierarchical clustering... ward : AgglomerativeClustering(n_clusters=6) ward.labels_ : [0 3 3 2 2 1 1 2 2 0 0 0 1 0 2 3 3 2 0 1 3 0 2 3 2 0 2 0 4 4 0 3 0 0 0 2 2 2 0 1 1 2 2 4 2 0 2 1 3 0 2 3 2 1 2 3 0 3 2 0 0 2 4 4 3 0 0 5 5 0 0 2 0 3 4 0 1 3 3 3 5 1 2 2 3 2 3 0 5 3 2 3 2 0 0 4 0 3 3 5 1 2 2 3 4 3 0 2 0 2 4 5 4 0 0 4 5 4 2 5 4 0 3 2 4 4 2 4 0 0 4 4 0 3 0 2 4 2 0 3 2 0 2] Elapsed time: 0.022815942764282227 graph_id used : 78166 - Beta version, working pretty good on 11-5-16 ! https://www.fotonower.com/velours/20480175,20480176,20480192,20480181,20480184,20480185,20480187,20480183,20480186,20480188,20480189,20480190,20480191?tags=mal_croppe,flou,pehd,pet_fonce,pet_clair,papier,carton,tetrapak,autre,environnement,refus,metal,film_plastique&datou_id_consolidate=4235&port_consolidate=20479443&tree_id=78166 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : velours_tree we use saveGeneral [1337123742, 1337122899] Looping around the photos to save general results len do output : 1 /20479443Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2579746') ('4234', '20479443', '1337123742', None, None, None, None, None, '2579746') ('4234', None, None, None, None, None, None, None, '2579746') ('4234', '20479443', '1337122899', None, None, None, None, None, '2579746') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.013316869735717773 save_final save missing photos in datou_result : time spend for datou_step_exec : 10.22272801399231 time spend to save output : 0.013683557510375977 total time spend for step 9 : 10.236411571502686 step10:send_mail_cod Wed Feb 12 16:22:41 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Qualipapia_P20479443_12-02-2025_16_22_41.pdf 20480157 imagette204801571739373761 20480158 imagette204801581739373761 20480174 imagette204801741739373761 20480163 imagette204801631739373761 20480166 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204801661739373761 20480167 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204801671739373761 20480169 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204801691739373762 20480165 imagette204801651739373763 20480168 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204801681739373763 20480171 imagette204801711739373764 20480172 imagette204801721739373764 20480173 imagette204801731739373764 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20479443 and hashtag_type = 4230 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20480175,20480176,20480192,20480181,20480184,20480185,20480187,20480183,20480186,20480188,20480189,20480190,20480191?tags=mal_croppe,flou,pehd,pet_fonce,pet_clair,papier,carton,tetrapak,autre,environnement,refus,metal,film_plastique&datou_id_consolidate=4235&port_consolidate=20479443&tree_id=78166 your option no_mail is active, we will not send the real mail to your client args[1337123742] : ((1337123742, -7.775476818455898, 492609224), (1337123742, -0.6095764118854592, 501862349), '0.22406120432686213') apple ((1337123742, -7.775476818455898, 492609224), (1337123742, -0.6095764118854592, 501862349), '0.22406120432686213') We are sending mail with results at cod@fotonower.com args[1337122899] : ((1337122899, -7.780035458387716, 492609224), (1337122899, -0.5526552109823938, 501862349), '0.22406120432686213') apple ((1337122899, -7.780035458387716, 492609224), (1337122899, -0.5526552109823938, 501862349), '0.22406120432686213') We are sending mail with results at cod@fotonower.com refus_total : 0.22406120432686213 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20479443 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20479443_12-02-2025_16_22_41.pdf results_Qualipapia_P20479443_12-02-2025_16_22_41.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20479443_12-02-2025_16_22_41.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4234','20479443','results_Qualipapia_P20479443_12-02-2025_16_22_41.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20479443_12-02-2025_16_22_41.pdf','pdf','','0.14','0.22406120432686213') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1337123742, 1337122899] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2579746') ('4234', '20479443', '1337123742', None, None, None, None, None, '2579746') ('4234', None, None, None, None, None, None, None, '2579746') ('4234', '20479443', '1337122899', None, None, None, None, None, '2579746') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 2 time used for this insertion : 0.011836051940917969 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.790215492248535 time spend to save output : 0.01203155517578125 total time spend for step 10 : 4.802247047424316 step11:split_time_score Wed Feb 12 16:22:46 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 3442, 'mtr_user_id': 31, 'name': 'classifieur_2camions_valcor_021122_v1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'deux_camions,camion_droite,camion_gauche,pas_de_camion', 'svm_portfolios_learning': '7659379,7659034,7657685,7657114', 'photo_hashtag_type': 4458, 'photo_desc_type': 5723, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107760533,2107760534,2107760535,2107760536'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('14', 2),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 12022025 20479443 Nombre de photos uploadées : 2 / 23040 (0%) 12022025 20479443 Nombre de photos taguées (types de déchets): 2 / 2 (100%) 12022025 20479443 Nombre de photos taguées (volume) : 0 / 2 (0%) elapsed_time : load_data_split_time_score 2.1457672119140625e-06 elapsed_time : order_list_meta_photo_and_scores 5.245208740234375e-06 ?? elapsed_time : fill_and_build_computed_from_old_data 0.0002238750457763672 elapsed_time : insert_dashboard_record_day_entry 0.023085594177246094 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.15829996426485274 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20470706_12-02-2025_10_17_40.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20470706 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20470706 AND mptpi.`type`=4230 To do Qualite : 0.2571924266909986 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20474222_12-02-2025_11_47_45.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20474222 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20474222 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476682 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476683 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476684 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476685 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476686 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476321 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476322 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476323 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476324 order by id desc limit 1 Qualite : 0.17840314144925185 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20476325_12-02-2025_12_46_55.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476325 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20476325 AND mptpi.`type`=4230 To do Qualite : 0.23342179416285125 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20476326_12-02-2025_12_36_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476326 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20476326 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476687 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476688 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476689 order by id desc limit 1 Qualite : 0.1924295939903281 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20476690_12-02-2025_13_32_08.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476690 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20476690 AND mptpi.`type`=4230 To do Qualite : 0.2119568776018645 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20476691_12-02-2025_13_23_51.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476691 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20476691 AND mptpi.`type`=4230 To do Qualite : 0.154875790367845 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20476692_12-02-2025_13_17_01.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20476692 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20476692 AND mptpi.`type`=4230 To do Qualite : 0.19008527192762648 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20477040_12-02-2025_13_51_47.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20477040 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20477040 AND mptpi.`type`=4230 To do Qualite : 0.19050715192060547 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20477041_12-02-2025_13_39_05.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20477041 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20477041 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20477469 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20477470 order by id desc limit 1 Qualite : 0.22219744748095124 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20477471_12-02-2025_15_33_55.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20477471 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20477471 AND mptpi.`type`=4230 To do Qualite : 0.213374648572307 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20477472_12-02-2025_14_47_07.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20477472 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20477472 AND mptpi.`type`=4230 To do Qualite : 0.2080923629286091 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20477473_12-02-2025_14_17_51.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20477473 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20477473 AND mptpi.`type`=4230 To do Qualite : 0.21079772282942882 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20477474_12-02-2025_14_32_10.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20477474 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20477474 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479190 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479191 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479192 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479193 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479194 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479195 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479196 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479197 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479199 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479200 order by id desc limit 1 Qualite : 0.20732146131397367 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20479201_12-02-2025_15_48_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479201 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20479201 AND mptpi.`type`=4230 To do Qualite : 0.22406120432686213 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20479443_12-02-2025_16_22_41.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479443 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20479443 AND mptpi.`type`=4230 To do Qualite : 0.1665337497911145 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20479444_12-02-2025_16_17_51.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479444 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20479444 AND mptpi.`type`=4230 To do Qualite : 0.14940220555870873 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20479445_12-02-2025_16_04_36.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20479445 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20479445 AND mptpi.`type`=4230 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'12022025': {'nb_upload': 2, 'nb_taggue_class': 2, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1337123742, 1337122899] Looping around the photos to save general results len do output : 1 /20479443Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2579746') ('4234', '20479443', '1337123742', None, None, None, None, None, '2579746') ('4234', None, None, None, None, None, None, None, '2579746') ('4234', '20479443', '1337122899', None, None, None, None, None, '2579746') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.012663602828979492 save_final save missing photos in datou_result : time spend for datou_step_exec : 19.597498893737793 time spend to save output : 0.012845993041992188 total time spend for step 11 : 19.610344886779785 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 1 len(input) + len(total_photo_id_missing) : 2 set_done_treatment 34.94user 23.14system 2:40.23elapsed 36%CPU (0avgtext+0avgdata 2457124maxresident)k 575352inputs+11120outputs (1292major+2275942minor)pagefaults 0swaps