python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 4234 -l 5 ' -s datou_current_4234 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 141569 load datou : 4234 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load THCL from format json or kwargs add thcl : 3237 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig add pdt : 5561 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4234, datou_cur_ids : ['2636547'] with mtr_portfolio_ids : ['21053699'] and first list_photo_ids : [] new path : /proc/141569/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, brightness, blur_detection, rle_unique_nms_with_priority, crop_condition, thcl, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , BFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 time to download the photos : 0.1587214469909668 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Wed Mar 5 09:20:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10151 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-03-05 09:20:32.225357: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-03-05 09:20:32.255080: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-03-05 09:20:32.257336: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f4918000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-03-05 09:20:32.257399: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-03-05 09:20:32.261568: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-03-05 09:20:32.512744: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xde206a0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-03-05 09:20:32.512805: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-03-05 09:20:32.514184: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-05 09:20:32.518274: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-05 09:20:32.522588: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-05 09:20:32.525509: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-05 09:20:32.526303: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-05 09:20:32.530096: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-05 09:20:32.531846: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-05 09:20:32.538537: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-05 09:20:32.540142: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-05 09:20:32.540236: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-05 09:20:32.541038: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-05 09:20:32.541059: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-05 09:20:32.541070: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-05 09:20:32.542377: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9399 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-03-05 09:20:32.900439: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-05 09:20:32.900579: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-05 09:20:32.900605: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-05 09:20:32.900627: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-05 09:20:32.900649: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-05 09:20:32.900670: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-05 09:20:32.900690: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-05 09:20:32.900712: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-05 09:20:32.902241: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-05 09:20:32.904238: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-05 09:20:32.904314: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-05 09:20:32.904346: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-05 09:20:32.904374: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-05 09:20:32.904402: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-05 09:20:32.904429: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-05 09:20:32.904455: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-05 09:20:32.904482: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-05 09:20:32.906639: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-05 09:20:32.906702: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-05 09:20:32.906718: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-05 09:20:32.906732: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-05 09:20:32.908843: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9399 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-03-05 09:20:41.906616: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-05 09:20:42.127350: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 1 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -113.76250 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 0.00014543533325195312 nb_pixel_total : 1090 time to create 1 rle with old method : 0.002215862274169922 length of segment : 33 time for calcul the mask position with numpy : 0.0001392364501953125 nb_pixel_total : 2401 time to create 1 rle with old method : 0.004720449447631836 length of segment : 76 time for calcul the mask position with numpy : 0.0001804828643798828 nb_pixel_total : 4189 time to create 1 rle with old method : 0.007361173629760742 length of segment : 97 time for calcul the mask position with numpy : 0.00011849403381347656 nb_pixel_total : 3157 time to create 1 rle with old method : 0.011217832565307617 length of segment : 88 time for calcul the mask position with numpy : 0.00011730194091796875 nb_pixel_total : 2003 time to create 1 rle with old method : 0.0028946399688720703 length of segment : 51 time for calcul the mask position with numpy : 0.0001499652862548828 nb_pixel_total : 2829 time to create 1 rle with old method : 0.005258321762084961 length of segment : 78 time for calcul the mask position with numpy : 7.605552673339844e-05 nb_pixel_total : 108 time to create 1 rle with old method : 0.000255584716796875 length of segment : 10 time for calcul the mask position with numpy : 5.030632019042969e-05 nb_pixel_total : 328 time to create 1 rle with old method : 0.0004894733428955078 length of segment : 25 time for calcul the mask position with numpy : 6.937980651855469e-05 nb_pixel_total : 917 time to create 1 rle with old method : 0.0013318061828613281 length of segment : 39 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -118.50469 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 16 time for calcul the mask position with numpy : 0.00010013580322265625 nb_pixel_total : 1739 time to create 1 rle with old method : 0.0025587081909179688 length of segment : 40 time for calcul the mask position with numpy : 8.7738037109375e-05 nb_pixel_total : 572 time to create 1 rle with old method : 0.0009794235229492188 length of segment : 24 time for calcul the mask position with numpy : 8.0108642578125e-05 nb_pixel_total : 1898 time to create 1 rle with old method : 0.002858877182006836 length of segment : 48 time for calcul the mask position with numpy : 6.0558319091796875e-05 nb_pixel_total : 424 time to create 1 rle with old method : 0.0008175373077392578 length of segment : 23 time for calcul the mask position with numpy : 5.0067901611328125e-05 nb_pixel_total : 140 time to create 1 rle with old method : 0.0002808570861816406 length of segment : 15 time for calcul the mask position with numpy : 3.504753112792969e-05 nb_pixel_total : 158 time to create 1 rle with old method : 0.0002932548522949219 length of segment : 10 time for calcul the mask position with numpy : 0.000125885009765625 nb_pixel_total : 2032 time to create 1 rle with old method : 0.0031995773315429688 length of segment : 55 time for calcul the mask position with numpy : 5.14984130859375e-05 nb_pixel_total : 347 time to create 1 rle with old method : 0.0005128383636474609 length of segment : 21 time for calcul the mask position with numpy : 4.839897155761719e-05 nb_pixel_total : 252 time to create 1 rle with old method : 0.00039768218994140625 length of segment : 17 time for calcul the mask position with numpy : 5.936622619628906e-05 nb_pixel_total : 1075 time to create 1 rle with old method : 0.001592397689819336 length of segment : 33 time for calcul the mask position with numpy : 5.5789947509765625e-05 nb_pixel_total : 587 time to create 1 rle with old method : 0.0008873939514160156 length of segment : 32 time for calcul the mask position with numpy : 7.343292236328125e-05 nb_pixel_total : 1468 time to create 1 rle with old method : 0.0023322105407714844 length of segment : 34 time for calcul the mask position with numpy : 4.863739013671875e-05 nb_pixel_total : 417 time to create 1 rle with old method : 0.0007138252258300781 length of segment : 24 time for calcul the mask position with numpy : 9.5367431640625e-05 nb_pixel_total : 1652 time to create 1 rle with old method : 0.0025186538696289062 length of segment : 36 time for calcul the mask position with numpy : 7.557868957519531e-05 nb_pixel_total : 1279 time to create 1 rle with old method : 0.0015244483947753906 length of segment : 108 time for calcul the mask position with numpy : 7.605552673339844e-05 nb_pixel_total : 1320 time to create 1 rle with old method : 0.001800537109375 length of segment : 35 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -118.81328 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 20 time for calcul the mask position with numpy : 7.963180541992188e-05 nb_pixel_total : 1627 time to create 1 rle with old method : 0.002023935317993164 length of segment : 50 time for calcul the mask position with numpy : 6.794929504394531e-05 nb_pixel_total : 1794 time to create 1 rle with old method : 0.0022170543670654297 length of segment : 52 time for calcul the mask position with numpy : 4.2438507080078125e-05 nb_pixel_total : 353 time to create 1 rle with old method : 0.0005018711090087891 length of segment : 25 time for calcul the mask position with numpy : 4.076957702636719e-05 nb_pixel_total : 532 time to create 1 rle with old method : 0.0008046627044677734 length of segment : 17 time for calcul the mask position with numpy : 7.62939453125e-05 nb_pixel_total : 1789 time to create 1 rle with old method : 0.0022957324981689453 length of segment : 68 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 433 time to create 1 rle with old method : 0.0006115436553955078 length of segment : 25 time for calcul the mask position with numpy : 5.459785461425781e-05 nb_pixel_total : 292 time to create 1 rle with old method : 0.00043010711669921875 length of segment : 24 time for calcul the mask position with numpy : 5.14984130859375e-05 nb_pixel_total : 831 time to create 1 rle with old method : 0.0011353492736816406 length of segment : 31 time for calcul the mask position with numpy : 5.0067901611328125e-05 nb_pixel_total : 722 time to create 1 rle with old method : 0.0009784698486328125 length of segment : 33 time for calcul the mask position with numpy : 5.030632019042969e-05 nb_pixel_total : 808 time to create 1 rle with old method : 0.0011737346649169922 length of segment : 37 time for calcul the mask position with numpy : 8.273124694824219e-05 nb_pixel_total : 2334 time to create 1 rle with old method : 0.003019571304321289 length of segment : 66 time for calcul the mask position with numpy : 4.172325134277344e-05 nb_pixel_total : 271 time to create 1 rle with old method : 0.00041556358337402344 length of segment : 23 time for calcul the mask position with numpy : 3.528594970703125e-05 nb_pixel_total : 181 time to create 1 rle with old method : 0.0003123283386230469 length of segment : 12 time for calcul the mask position with numpy : 3.6716461181640625e-05 nb_pixel_total : 290 time to create 1 rle with old method : 0.0004620552062988281 length of segment : 19 time for calcul the mask position with numpy : 4.57763671875e-05 nb_pixel_total : 737 time to create 1 rle with old method : 0.0010972023010253906 length of segment : 32 time for calcul the mask position with numpy : 0.0001220703125 nb_pixel_total : 4122 time to create 1 rle with old method : 0.005165815353393555 length of segment : 84 time for calcul the mask position with numpy : 5.53131103515625e-05 nb_pixel_total : 564 time to create 1 rle with old method : 0.0007486343383789062 length of segment : 40 time for calcul the mask position with numpy : 4.482269287109375e-05 nb_pixel_total : 429 time to create 1 rle with old method : 0.0006308555603027344 length of segment : 24 time for calcul the mask position with numpy : 4.363059997558594e-05 nb_pixel_total : 692 time to create 1 rle with old method : 0.0009391307830810547 length of segment : 24 time for calcul the mask position with numpy : 4.7206878662109375e-05 nb_pixel_total : 727 time to create 1 rle with old method : 0.0009908676147460938 length of segment : 26 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.35234 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 5.1021575927734375e-05 nb_pixel_total : 425 time to create 1 rle with old method : 0.0006415843963623047 length of segment : 26 time for calcul the mask position with numpy : 3.814697265625e-05 nb_pixel_total : 187 time to create 1 rle with old method : 0.0003018379211425781 length of segment : 15 time for calcul the mask position with numpy : 4.172325134277344e-05 nb_pixel_total : 474 time to create 1 rle with old method : 0.0006847381591796875 length of segment : 26 time for calcul the mask position with numpy : 5.626678466796875e-05 nb_pixel_total : 409 time to create 1 rle with old method : 0.000629425048828125 length of segment : 41 time for calcul the mask position with numpy : 5.269050598144531e-05 nb_pixel_total : 520 time to create 1 rle with old method : 0.0008654594421386719 length of segment : 57 time for calcul the mask position with numpy : 7.152557373046875e-05 nb_pixel_total : 1999 time to create 1 rle with old method : 0.002579927444458008 length of segment : 59 time for calcul the mask position with numpy : 4.7206878662109375e-05 nb_pixel_total : 366 time to create 1 rle with old method : 0.0005431175231933594 length of segment : 23 time for calcul the mask position with numpy : 4.6253204345703125e-05 nb_pixel_total : 506 time to create 1 rle with old method : 0.0008530616760253906 length of segment : 27 Processing 1 images image shape: (400, 400, 3) min: 85.00000 max: 237.00000 molded_images shape: (1, 640, 640, 3) min: -38.70000 max: 124.53359 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 20.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -87.44609 max: 137.74062 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 160, 3) min: 21.00000 max: 238.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 124.85000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 5.221366882324219e-05 nb_pixel_total : 384 time to create 1 rle with old method : 0.0005831718444824219 length of segment : 33 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -119.45000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 4.6253204345703125e-05 nb_pixel_total : 306 time to create 1 rle with old method : 0.0004639625549316406 length of segment : 26 time for calcul the mask position with numpy : 5.14984130859375e-05 nb_pixel_total : 730 time to create 1 rle with old method : 0.0010225772857666016 length of segment : 32 time for calcul the mask position with numpy : 3.933906555175781e-05 nb_pixel_total : 289 time to create 1 rle with old method : 0.00042057037353515625 length of segment : 25 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -111.53594 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 19 time for calcul the mask position with numpy : 0.00014781951904296875 nb_pixel_total : 877 time to create 1 rle with old method : 0.0018627643585205078 length of segment : 48 time for calcul the mask position with numpy : 7.343292236328125e-05 nb_pixel_total : 707 time to create 1 rle with old method : 0.0011775493621826172 length of segment : 51 time for calcul the mask position with numpy : 7.081031799316406e-05 nb_pixel_total : 542 time to create 1 rle with old method : 0.0010464191436767578 length of segment : 43 time for calcul the mask position with numpy : 0.00014710426330566406 nb_pixel_total : 7147 time to create 1 rle with old method : 0.00932455062866211 length of segment : 81 time for calcul the mask position with numpy : 6.103515625e-05 nb_pixel_total : 1099 time to create 1 rle with old method : 0.0014183521270751953 length of segment : 42 time for calcul the mask position with numpy : 5.817413330078125e-05 nb_pixel_total : 1436 time to create 1 rle with old method : 0.0018811225891113281 length of segment : 57 time for calcul the mask position with numpy : 7.224082946777344e-05 nb_pixel_total : 1641 time to create 1 rle with old method : 0.0021839141845703125 length of segment : 63 time for calcul the mask position with numpy : 4.0531158447265625e-05 nb_pixel_total : 190 time to create 1 rle with old method : 0.00031113624572753906 length of segment : 10 time for calcul the mask position with numpy : 3.910064697265625e-05 nb_pixel_total : 108 time to create 1 rle with old method : 0.00020194053649902344 length of segment : 9 time for calcul the mask position with numpy : 7.62939453125e-05 nb_pixel_total : 2406 time to create 1 rle with old method : 0.0029833316802978516 length of segment : 74 time for calcul the mask position with numpy : 5.698204040527344e-05 nb_pixel_total : 175 time to create 1 rle with old method : 0.0002732276916503906 length of segment : 21 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 523 time to create 1 rle with old method : 0.0007131099700927734 length of segment : 23 time for calcul the mask position with numpy : 0.00018310546875 nb_pixel_total : 7271 time to create 1 rle with old method : 0.008644819259643555 length of segment : 81 time for calcul the mask position with numpy : 7.677078247070312e-05 nb_pixel_total : 1127 time to create 1 rle with old method : 0.001516580581665039 length of segment : 48 time for calcul the mask position with numpy : 6.4849853515625e-05 nb_pixel_total : 1101 time to create 1 rle with old method : 0.0014238357543945312 length of segment : 47 time for calcul the mask position with numpy : 3.719329833984375e-05 nb_pixel_total : 68 time to create 1 rle with old method : 0.0001270771026611328 length of segment : 15 time for calcul the mask position with numpy : 5.054473876953125e-05 nb_pixel_total : 273 time to create 1 rle with old method : 0.00043702125549316406 length of segment : 41 time for calcul the mask position with numpy : 4.744529724121094e-05 nb_pixel_total : 402 time to create 1 rle with old method : 0.0005724430084228516 length of segment : 30 time for calcul the mask position with numpy : 7.414817810058594e-05 nb_pixel_total : 2061 time to create 1 rle with old method : 0.0026426315307617188 length of segment : 49 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -121.64922 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 26 time for calcul the mask position with numpy : 0.00010418891906738281 nb_pixel_total : 914 time to create 1 rle with old method : 0.001207113265991211 length of segment : 61 time for calcul the mask position with numpy : 5.817413330078125e-05 nb_pixel_total : 361 time to create 1 rle with old method : 0.0005624294281005859 length of segment : 21 time for calcul the mask position with numpy : 4.38690185546875e-05 nb_pixel_total : 138 time to create 1 rle with old method : 0.00025391578674316406 length of segment : 12 time for calcul the mask position with numpy : 4.100799560546875e-05 nb_pixel_total : 104 time to create 1 rle with old method : 0.00025391578674316406 length of segment : 11 time for calcul the mask position with numpy : 6.914138793945312e-05 nb_pixel_total : 996 time to create 1 rle with old method : 0.0014646053314208984 length of segment : 41 time for calcul the mask position with numpy : 7.176399230957031e-05 nb_pixel_total : 940 time to create 1 rle with old method : 0.0014598369598388672 length of segment : 46 time for calcul the mask position with numpy : 3.910064697265625e-05 nb_pixel_total : 256 time to create 1 rle with old method : 0.0003521442413330078 length of segment : 25 time for calcul the mask position with numpy : 6.532669067382812e-05 nb_pixel_total : 1936 time to create 1 rle with old method : 0.002733945846557617 length of segment : 51 time for calcul the mask position with numpy : 7.224082946777344e-05 nb_pixel_total : 1091 time to create 1 rle with old method : 0.0014462471008300781 length of segment : 48 time for calcul the mask position with numpy : 7.009506225585938e-05 nb_pixel_total : 807 time to create 1 rle with old method : 0.0012021064758300781 length of segment : 42 time for calcul the mask position with numpy : 5.8650970458984375e-05 nb_pixel_total : 188 time to create 1 rle with old method : 0.0003688335418701172 length of segment : 43 time for calcul the mask position with numpy : 4.863739013671875e-05 nb_pixel_total : 84 time to create 1 rle with old method : 0.00021123886108398438 length of segment : 10 time for calcul the mask position with numpy : 6.0558319091796875e-05 nb_pixel_total : 794 time to create 1 rle with old method : 0.0011801719665527344 length of segment : 27 time for calcul the mask position with numpy : 5.0067901611328125e-05 nb_pixel_total : 240 time to create 1 rle with old method : 0.0004062652587890625 length of segment : 18 time for calcul the mask position with numpy : 7.43865966796875e-05 nb_pixel_total : 1435 time to create 1 rle with old method : 0.0023996829986572266 length of segment : 34 time for calcul the mask position with numpy : 0.0001342296600341797 nb_pixel_total : 904 time to create 1 rle with old method : 0.0013763904571533203 length of segment : 50 time for calcul the mask position with numpy : 6.771087646484375e-05 nb_pixel_total : 650 time to create 1 rle with old method : 0.0009961128234863281 length of segment : 23 time for calcul the mask position with numpy : 9.560585021972656e-05 nb_pixel_total : 1949 time to create 1 rle with old method : 0.0029535293579101562 length of segment : 48 time for calcul the mask position with numpy : 7.05718994140625e-05 nb_pixel_total : 786 time to create 1 rle with old method : 0.0012638568878173828 length of segment : 24 time for calcul the mask position with numpy : 5.841255187988281e-05 nb_pixel_total : 178 time to create 1 rle with old method : 0.00026917457580566406 length of segment : 20 time for calcul the mask position with numpy : 4.601478576660156e-05 nb_pixel_total : 276 time to create 1 rle with old method : 0.0004515647888183594 length of segment : 19 time for calcul the mask position with numpy : 5.507469177246094e-05 nb_pixel_total : 436 time to create 1 rle with old method : 0.0006227493286132812 length of segment : 23 time for calcul the mask position with numpy : 4.935264587402344e-05 nb_pixel_total : 162 time to create 1 rle with old method : 0.0002663135528564453 length of segment : 21 time for calcul the mask position with numpy : 7.677078247070312e-05 nb_pixel_total : 2069 time to create 1 rle with old method : 0.0028841495513916016 length of segment : 48 time for calcul the mask position with numpy : 8.130073547363281e-05 nb_pixel_total : 1374 time to create 1 rle with old method : 0.0019643306732177734 length of segment : 42 time for calcul the mask position with numpy : 0.00013136863708496094 nb_pixel_total : 4005 time to create 1 rle with old method : 0.005499124526977539 length of segment : 96 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -114.93047 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 21 time for calcul the mask position with numpy : 9.679794311523438e-05 nb_pixel_total : 561 time to create 1 rle with old method : 0.0008203983306884766 length of segment : 23 time for calcul the mask position with numpy : 4.410743713378906e-05 nb_pixel_total : 240 time to create 1 rle with old method : 0.00037217140197753906 length of segment : 16 time for calcul the mask position with numpy : 5.4836273193359375e-05 nb_pixel_total : 276 time to create 1 rle with old method : 0.00039458274841308594 length of segment : 24 time for calcul the mask position with numpy : 4.0531158447265625e-05 nb_pixel_total : 337 time to create 1 rle with old method : 0.0004687309265136719 length of segment : 22 time for calcul the mask position with numpy : 3.6716461181640625e-05 nb_pixel_total : 246 time to create 1 rle with old method : 0.0003638267517089844 length of segment : 16 time for calcul the mask position with numpy : 4.38690185546875e-05 nb_pixel_total : 492 time to create 1 rle with old method : 0.0006763935089111328 length of segment : 22 time for calcul the mask position with numpy : 4.172325134277344e-05 nb_pixel_total : 241 time to create 1 rle with old method : 0.0003581047058105469 length of segment : 17 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 144 time to create 1 rle with old method : 0.00023865699768066406 length of segment : 13 time for calcul the mask position with numpy : 5.1021575927734375e-05 nb_pixel_total : 482 time to create 1 rle with old method : 0.0006377696990966797 length of segment : 44 time for calcul the mask position with numpy : 7.891654968261719e-05 nb_pixel_total : 1518 time to create 1 rle with old method : 0.001984119415283203 length of segment : 63 time for calcul the mask position with numpy : 9.584426879882812e-05 nb_pixel_total : 256 time to create 1 rle with old method : 0.0005521774291992188 length of segment : 19 time for calcul the mask position with numpy : 4.9114227294921875e-05 nb_pixel_total : 333 time to create 1 rle with old method : 0.00045037269592285156 length of segment : 33 time for calcul the mask position with numpy : 6.628036499023438e-05 nb_pixel_total : 185 time to create 1 rle with old method : 0.00037670135498046875 length of segment : 17 time for calcul the mask position with numpy : 6.246566772460938e-05 nb_pixel_total : 1185 time to create 1 rle with old method : 0.0016870498657226562 length of segment : 25 time for calcul the mask position with numpy : 5.221366882324219e-05 nb_pixel_total : 112 time to create 1 rle with old method : 0.00034928321838378906 length of segment : 12 time for calcul the mask position with numpy : 7.009506225585938e-05 nb_pixel_total : 175 time to create 1 rle with old method : 0.0003528594970703125 length of segment : 14 time for calcul the mask position with numpy : 5.340576171875e-05 nb_pixel_total : 213 time to create 1 rle with old method : 0.0004761219024658203 length of segment : 16 time for calcul the mask position with numpy : 4.696846008300781e-05 nb_pixel_total : 350 time to create 1 rle with old method : 0.0004792213439941406 length of segment : 28 time for calcul the mask position with numpy : 5.91278076171875e-05 nb_pixel_total : 826 time to create 1 rle with old method : 0.0011425018310546875 length of segment : 48 time for calcul the mask position with numpy : 4.649162292480469e-05 nb_pixel_total : 228 time to create 1 rle with old method : 0.00035500526428222656 length of segment : 19 time for calcul the mask position with numpy : 5.793571472167969e-05 nb_pixel_total : 1098 time to create 1 rle with old method : 0.00153350830078125 length of segment : 25 Processing 1 images image shape: (400, 400, 3) min: 83.00000 max: 236.00000 molded_images shape: (1, 640, 640, 3) min: -40.70000 max: 124.65469 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 16.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -99.44219 max: 149.52578 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 12 time for calcul the mask position with numpy : 9.083747863769531e-05 nb_pixel_total : 606 time to create 1 rle with old method : 0.0009119510650634766 length of segment : 25 time for calcul the mask position with numpy : 7.43865966796875e-05 nb_pixel_total : 1396 time to create 1 rle with old method : 0.0019350051879882812 length of segment : 34 time for calcul the mask position with numpy : 6.628036499023438e-05 nb_pixel_total : 312 time to create 1 rle with old method : 0.0004601478576660156 length of segment : 33 time for calcul the mask position with numpy : 0.00011420249938964844 nb_pixel_total : 2390 time to create 1 rle with old method : 0.0029985904693603516 length of segment : 53 time for calcul the mask position with numpy : 4.57763671875e-05 nb_pixel_total : 283 time to create 1 rle with old method : 0.00040459632873535156 length of segment : 23 time for calcul the mask position with numpy : 9.489059448242188e-05 nb_pixel_total : 2540 time to create 1 rle with old method : 0.0033104419708251953 length of segment : 51 time for calcul the mask position with numpy : 4.601478576660156e-05 nb_pixel_total : 176 time to create 1 rle with old method : 0.0002684593200683594 length of segment : 17 time for calcul the mask position with numpy : 6.747245788574219e-05 nb_pixel_total : 1587 time to create 1 rle with old method : 0.0020952224731445312 length of segment : 49 time for calcul the mask position with numpy : 4.553794860839844e-05 nb_pixel_total : 282 time to create 1 rle with old method : 0.000476837158203125 length of segment : 23 time for calcul the mask position with numpy : 0.00011157989501953125 nb_pixel_total : 2395 time to create 1 rle with old method : 0.0034656524658203125 length of segment : 52 time for calcul the mask position with numpy : 6.723403930664062e-05 nb_pixel_total : 421 time to create 1 rle with old method : 0.0006418228149414062 length of segment : 37 time for calcul the mask position with numpy : 8.058547973632812e-05 nb_pixel_total : 1394 time to create 1 rle with old method : 0.0019049644470214844 length of segment : 34 Processing 1 images image shape: (400, 160, 3) min: 22.00000 max: 252.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 134.08047 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 4.601478576660156e-05 nb_pixel_total : 322 time to create 1 rle with old method : 0.00044727325439453125 length of segment : 36 time for calcul the mask position with numpy : 4.863739013671875e-05 nb_pixel_total : 457 time to create 1 rle with old method : 0.0006465911865234375 length of segment : 20 time for calcul the mask position with numpy : 4.6253204345703125e-05 nb_pixel_total : 275 time to create 1 rle with old method : 0.0004439353942871094 length of segment : 17 Processing 1 images image shape: (400, 400, 3) min: 2.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -109.98906 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 7.367134094238281e-05 nb_pixel_total : 1329 time to create 1 rle with old method : 0.001691579818725586 length of segment : 95 time for calcul the mask position with numpy : 5.245208740234375e-05 nb_pixel_total : 557 time to create 1 rle with old method : 0.0008141994476318359 length of segment : 39 time for calcul the mask position with numpy : 5.1975250244140625e-05 nb_pixel_total : 283 time to create 1 rle with old method : 0.0004508495330810547 length of segment : 21 time for calcul the mask position with numpy : 0.00010728836059570312 nb_pixel_total : 1879 time to create 1 rle with old method : 0.0027136802673339844 length of segment : 146 time for calcul the mask position with numpy : 6.794929504394531e-05 nb_pixel_total : 888 time to create 1 rle with old method : 0.0013475418090820312 length of segment : 56 time for calcul the mask position with numpy : 7.295608520507812e-05 nb_pixel_total : 1433 time to create 1 rle with old method : 0.0018374919891357422 length of segment : 60 time for calcul the mask position with numpy : 0.00011873245239257812 nb_pixel_total : 2989 time to create 1 rle with old method : 0.004024982452392578 length of segment : 139 time for calcul the mask position with numpy : 0.00015425682067871094 nb_pixel_total : 2687 time to create 1 rle with old method : 0.0037627220153808594 length of segment : 138 time for calcul the mask position with numpy : 0.000141143798828125 nb_pixel_total : 3441 time to create 1 rle with old method : 0.004586219787597656 length of segment : 139 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -109.11797 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 7.128715515136719e-05 nb_pixel_total : 352 time to create 1 rle with old method : 0.0007002353668212891 length of segment : 28 time for calcul the mask position with numpy : 6.985664367675781e-05 nb_pixel_total : 181 time to create 1 rle with old method : 0.0003743171691894531 length of segment : 17 time for calcul the mask position with numpy : 6.890296936035156e-05 nb_pixel_total : 953 time to create 1 rle with old method : 0.0018494129180908203 length of segment : 35 time for calcul the mask position with numpy : 6.866455078125e-05 nb_pixel_total : 289 time to create 1 rle with old method : 0.0007359981536865234 length of segment : 45 time for calcul the mask position with numpy : 0.0001308917999267578 nb_pixel_total : 3088 time to create 1 rle with old method : 0.005673646926879883 length of segment : 136 time for calcul the mask position with numpy : 9.608268737792969e-05 nb_pixel_total : 356 time to create 1 rle with old method : 0.0008709430694580078 length of segment : 22 time for calcul the mask position with numpy : 0.00012636184692382812 nb_pixel_total : 1928 time to create 1 rle with old method : 0.0036232471466064453 length of segment : 64 time for calcul the mask position with numpy : 6.985664367675781e-05 nb_pixel_total : 383 time to create 1 rle with old method : 0.0008492469787597656 length of segment : 20 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -110.64141 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 17 time for calcul the mask position with numpy : 9.608268737792969e-05 nb_pixel_total : 256 time to create 1 rle with old method : 0.0007340908050537109 length of segment : 13 time for calcul the mask position with numpy : 9.846687316894531e-05 nb_pixel_total : 690 time to create 1 rle with old method : 0.0015971660614013672 length of segment : 45 time for calcul the mask position with numpy : 8.344650268554688e-05 nb_pixel_total : 442 time to create 1 rle with old method : 0.0010118484497070312 length of segment : 40 time for calcul the mask position with numpy : 6.461143493652344e-05 nb_pixel_total : 166 time to create 1 rle with old method : 0.0004372596740722656 length of segment : 27 time for calcul the mask position with numpy : 6.914138793945312e-05 nb_pixel_total : 863 time to create 1 rle with old method : 0.0020782947540283203 length of segment : 24 time for calcul the mask position with numpy : 6.127357482910156e-05 nb_pixel_total : 278 time to create 1 rle with old method : 0.0007092952728271484 length of segment : 14 time for calcul the mask position with numpy : 5.0067901611328125e-05 nb_pixel_total : 91 time to create 1 rle with old method : 0.00020074844360351562 length of segment : 16 time for calcul the mask position with numpy : 5.2928924560546875e-05 nb_pixel_total : 96 time to create 1 rle with old method : 0.00023412704467773438 length of segment : 31 time for calcul the mask position with numpy : 0.00014090538024902344 nb_pixel_total : 3542 time to create 1 rle with old method : 0.006228446960449219 length of segment : 61 time for calcul the mask position with numpy : 0.00010323524475097656 nb_pixel_total : 689 time to create 1 rle with old method : 0.0016126632690429688 length of segment : 46 time for calcul the mask position with numpy : 6.937980651855469e-05 nb_pixel_total : 198 time to create 1 rle with old method : 0.0005710124969482422 length of segment : 19 time for calcul the mask position with numpy : 7.271766662597656e-05 nb_pixel_total : 245 time to create 1 rle with old method : 0.0006361007690429688 length of segment : 17 time for calcul the mask position with numpy : 5.078315734863281e-05 nb_pixel_total : 245 time to create 1 rle with old method : 0.00041985511779785156 length of segment : 32 time for calcul the mask position with numpy : 5.2928924560546875e-05 nb_pixel_total : 630 time to create 1 rle with old method : 0.000843048095703125 length of segment : 34 time for calcul the mask position with numpy : 6.651878356933594e-05 nb_pixel_total : 1271 time to create 1 rle with old method : 0.0016245841979980469 length of segment : 44 time for calcul the mask position with numpy : 5.7220458984375e-05 nb_pixel_total : 653 time to create 1 rle with old method : 0.0009338855743408203 length of segment : 37 time for calcul the mask position with numpy : 9.083747863769531e-05 nb_pixel_total : 1154 time to create 1 rle with old method : 0.0015196800231933594 length of segment : 46 Processing 1 images image shape: (400, 400, 3) min: 34.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -81.83672 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 0.00010633468627929688 nb_pixel_total : 477 time to create 1 rle with old method : 0.0007543563842773438 length of segment : 26 time for calcul the mask position with numpy : 6.127357482910156e-05 nb_pixel_total : 256 time to create 1 rle with old method : 0.00043463706970214844 length of segment : 19 time for calcul the mask position with numpy : 6.103515625e-05 nb_pixel_total : 253 time to create 1 rle with old method : 0.0003857612609863281 length of segment : 20 time for calcul the mask position with numpy : 5.173683166503906e-05 nb_pixel_total : 482 time to create 1 rle with old method : 0.000682830810546875 length of segment : 23 time for calcul the mask position with numpy : 6.437301635742188e-05 nb_pixel_total : 307 time to create 1 rle with old method : 0.00046944618225097656 length of segment : 21 time for calcul the mask position with numpy : 8.96453857421875e-05 nb_pixel_total : 2123 time to create 1 rle with old method : 0.0028035640716552734 length of segment : 42 time for calcul the mask position with numpy : 5.435943603515625e-05 nb_pixel_total : 322 time to create 1 rle with old method : 0.00045752525329589844 length of segment : 27 time for calcul the mask position with numpy : 6.222724914550781e-05 nb_pixel_total : 715 time to create 1 rle with old method : 0.0009403228759765625 length of segment : 36 time for calcul the mask position with numpy : 8.893013000488281e-05 nb_pixel_total : 2263 time to create 1 rle with old method : 0.003039121627807617 length of segment : 44 Processing 1 images image shape: (400, 400, 3) min: 87.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -34.57891 max: 146.26406 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.46172 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 12 time for calcul the mask position with numpy : 0.00015115737915039062 nb_pixel_total : 895 time to create 1 rle with old method : 0.0018925666809082031 length of segment : 40 time for calcul the mask position with numpy : 7.152557373046875e-05 nb_pixel_total : 169 time to create 1 rle with old method : 0.0004506111145019531 length of segment : 14 time for calcul the mask position with numpy : 7.295608520507812e-05 nb_pixel_total : 300 time to create 1 rle with old method : 0.0006728172302246094 length of segment : 22 time for calcul the mask position with numpy : 8.869171142578125e-05 nb_pixel_total : 617 time to create 1 rle with old method : 0.0012784004211425781 length of segment : 43 time for calcul the mask position with numpy : 0.0001049041748046875 nb_pixel_total : 1440 time to create 1 rle with old method : 0.0029091835021972656 length of segment : 46 time for calcul the mask position with numpy : 8.153915405273438e-05 nb_pixel_total : 587 time to create 1 rle with old method : 0.0012521743774414062 length of segment : 36 time for calcul the mask position with numpy : 6.198883056640625e-05 nb_pixel_total : 227 time to create 1 rle with old method : 0.0005521774291992188 length of segment : 17 time for calcul the mask position with numpy : 0.00013113021850585938 nb_pixel_total : 1635 time to create 1 rle with old method : 0.00600433349609375 length of segment : 55 time for calcul the mask position with numpy : 6.771087646484375e-05 nb_pixel_total : 220 time to create 1 rle with old method : 0.0004973411560058594 length of segment : 26 time for calcul the mask position with numpy : 0.00016498565673828125 nb_pixel_total : 1167 time to create 1 rle with old method : 0.0024733543395996094 length of segment : 59 time for calcul the mask position with numpy : 6.29425048828125e-05 nb_pixel_total : 221 time to create 1 rle with old method : 0.0005242824554443359 length of segment : 17 time for calcul the mask position with numpy : 5.412101745605469e-05 nb_pixel_total : 122 time to create 1 rle with old method : 0.00030803680419921875 length of segment : 15 Processing 1 images image shape: (400, 160, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 143.77578 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 7.033348083496094e-05 nb_pixel_total : 678 time to create 1 rle with old method : 0.0013773441314697266 length of segment : 51 time for calcul the mask position with numpy : 6.103515625e-05 nb_pixel_total : 468 time to create 1 rle with old method : 0.0011959075927734375 length of segment : 37 time for calcul the mask position with numpy : 7.653236389160156e-05 nb_pixel_total : 487 time to create 1 rle with old method : 0.0010857582092285156 length of segment : 37 time for calcul the mask position with numpy : 5.91278076171875e-05 nb_pixel_total : 46 time to create 1 rle with old method : 0.0001461505889892578 length of segment : 11 time for calcul the mask position with numpy : 5.054473876953125e-05 nb_pixel_total : 106 time to create 1 rle with old method : 0.00026702880859375 length of segment : 13 time for calcul the mask position with numpy : 5.53131103515625e-05 nb_pixel_total : 133 time to create 1 rle with old method : 0.0003237724304199219 length of segment : 17 Processing 1 images image shape: (240, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 147.54141 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 8.893013000488281e-05 nb_pixel_total : 1739 time to create 1 rle with old method : 0.0035157203674316406 length of segment : 44 time for calcul the mask position with numpy : 0.0001494884490966797 nb_pixel_total : 3969 time to create 1 rle with old method : 0.007803678512573242 length of segment : 97 time for calcul the mask position with numpy : 0.0003204345703125 nb_pixel_total : 9683 time to create 1 rle with old method : 0.020077228546142578 length of segment : 264 time for calcul the mask position with numpy : 8.893013000488281e-05 nb_pixel_total : 1702 time to create 1 rle with old method : 0.0021915435791015625 length of segment : 43 Processing 1 images image shape: (240, 400, 3) min: 2.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 149.91250 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (240, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.74844 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 13 time for calcul the mask position with numpy : 7.43865966796875e-05 nb_pixel_total : 1246 time to create 1 rle with old method : 0.001653909683227539 length of segment : 41 time for calcul the mask position with numpy : 4.673004150390625e-05 nb_pixel_total : 196 time to create 1 rle with old method : 0.00032639503479003906 length of segment : 15 time for calcul the mask position with numpy : 3.743171691894531e-05 nb_pixel_total : 227 time to create 1 rle with old method : 0.00035858154296875 length of segment : 19 time for calcul the mask position with numpy : 4.458427429199219e-05 nb_pixel_total : 669 time to create 1 rle with old method : 0.000926971435546875 length of segment : 26 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 350 time to create 1 rle with old method : 0.0004818439483642578 length of segment : 22 time for calcul the mask position with numpy : 5.14984130859375e-05 nb_pixel_total : 813 time to create 1 rle with old method : 0.001088857650756836 length of segment : 29 time for calcul the mask position with numpy : 3.838539123535156e-05 nb_pixel_total : 249 time to create 1 rle with old method : 0.0003769397735595703 length of segment : 21 time for calcul the mask position with numpy : 5.745887756347656e-05 nb_pixel_total : 1113 time to create 1 rle with old method : 0.0014498233795166016 length of segment : 39 time for calcul the mask position with numpy : 4.2438507080078125e-05 nb_pixel_total : 400 time to create 1 rle with old method : 0.0006103515625 length of segment : 19 time for calcul the mask position with numpy : 5.2928924560546875e-05 nb_pixel_total : 519 time to create 1 rle with old method : 0.000762939453125 length of segment : 74 time for calcul the mask position with numpy : 5.364418029785156e-05 nb_pixel_total : 149 time to create 1 rle with old method : 0.0002579689025878906 length of segment : 12 time for calcul the mask position with numpy : 6.29425048828125e-05 nb_pixel_total : 295 time to create 1 rle with old method : 0.0004513263702392578 length of segment : 27 time for calcul the mask position with numpy : 7.295608520507812e-05 nb_pixel_total : 485 time to create 1 rle with old method : 0.0006945133209228516 length of segment : 26 Processing 1 images image shape: (240, 400, 3) min: 46.00000 max: 251.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 143.61562 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 4.410743713378906e-05 nb_pixel_total : 594 time to create 1 rle with old method : 0.0008528232574462891 length of segment : 24 Processing 1 images image shape: (240, 400, 3) min: 101.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 146.35781 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (240, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 10 time for calcul the mask position with numpy : 4.6253204345703125e-05 nb_pixel_total : 440 time to create 1 rle with old method : 0.0005748271942138672 length of segment : 41 time for calcul the mask position with numpy : 3.719329833984375e-05 nb_pixel_total : 167 time to create 1 rle with old method : 0.0002532005310058594 length of segment : 18 time for calcul the mask position with numpy : 4.315376281738281e-05 nb_pixel_total : 564 time to create 1 rle with old method : 0.0007526874542236328 length of segment : 33 time for calcul the mask position with numpy : 4.792213439941406e-05 nb_pixel_total : 367 time to create 1 rle with old method : 0.0005283355712890625 length of segment : 27 time for calcul the mask position with numpy : 4.029273986816406e-05 nb_pixel_total : 109 time to create 1 rle with old method : 0.00016427040100097656 length of segment : 21 time for calcul the mask position with numpy : 4.315376281738281e-05 nb_pixel_total : 391 time to create 1 rle with old method : 0.0005578994750976562 length of segment : 27 time for calcul the mask position with numpy : 4.410743713378906e-05 nb_pixel_total : 181 time to create 1 rle with old method : 0.0002868175506591797 length of segment : 16 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 358 time to create 1 rle with old method : 0.0005216598510742188 length of segment : 26 time for calcul the mask position with numpy : 0.0003094673156738281 nb_pixel_total : 22338 time to create 1 rle with old method : 0.025386810302734375 length of segment : 154 time for calcul the mask position with numpy : 0.0003809928894042969 nb_pixel_total : 24378 time to create 1 rle with old method : 0.03015422821044922 length of segment : 132 Processing 1 images image shape: (240, 160, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 138.53684 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Detection mask done ! Trying to reset tf kernel 142036 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 3091 tf kernel not reseted sub process len(results) : 227 len(list_Values) 227 None max_time_sub_proc : 3600 parent process len(results) : 0 len(list_Values) 227 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 8190 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Catched exception ! Connect or reconnect ! Number saved : None batch 1 Loaded 227 chid ids of type : 4228 Number RLEs to save : 8808 save missing photos in datou_result : time spend for datou_step_exec : 25.103167295455933 time spend to save output : 0.7487502098083496 total time spend for step 1 : 25.851917505264282 step2:brightness Wed Mar 5 09:20:54 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1741162828_141569_1340965857_40fa79da36149d6e1e0719f204af2cd6.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 1 time used for this insertion : 0.011542558670043945 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 1 time used for this insertion : 0.011206388473510742 save missing photos in datou_result : time spend for datou_step_exec : 0.567373514175415 time spend to save output : 0.02693343162536621 total time spend for step 2 : 0.5943069458007812 step3:blur_detection Wed Mar 5 09:20:55 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1741162828_141569_1340965857_40fa79da36149d6e1e0719f204af2cd6.jpg resize: (1440, 2560) 1340965857 -7.225758406722352 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 1 time used for this insertion : 0.012896299362182617 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 1 time used for this insertion : 0.011799335479736328 save missing photos in datou_result : time spend for datou_step_exec : 2.374497890472412 time spend to save output : 0.02963995933532715 total time spend for step 3 : 2.4041378498077393 step4:rle_unique_nms_with_priority Wed Mar 5 09:20:57 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 227 chid ids of type : 4228 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 227 nb_hashtags : 7 time to prepare the origin masks : 1.9128780364990234 time for calcul the mask position with numpy : 0.25550031661987305 nb_pixel_total : 3484637 time to create 1 rle with new method : 0.34651780128479004 time for calcul the mask position with numpy : 0.015798568725585938 nb_pixel_total : 384 time to create 1 rle with old method : 0.0005011558532714844 time for calcul the mask position with numpy : 0.016196012496948242 nb_pixel_total : 158 time to create 1 rle with old method : 0.000202178955078125 time for calcul the mask position with numpy : 0.015431880950927734 nb_pixel_total : 108 time to create 1 rle with old method : 0.00017118453979492188 time for calcul the mask position with numpy : 0.015961647033691406 nb_pixel_total : 424 time to create 1 rle with old method : 0.0005970001220703125 time for calcul the mask position with numpy : 0.015966176986694336 nb_pixel_total : 417 time to create 1 rle with old method : 0.0005331039428710938 time for calcul the mask position with numpy : 0.015681982040405273 nb_pixel_total : 4122 time to create 1 rle with old method : 0.004759073257446289 time for calcul the mask position with numpy : 0.015850543975830078 nb_pixel_total : 2401 time to create 1 rle with old method : 0.002762317657470703 time for calcul the mask position with numpy : 0.015996456146240234 nb_pixel_total : 1627 time to create 1 rle with old method : 0.0019922256469726562 time for calcul the mask position with numpy : 0.016620397567749023 nb_pixel_total : 290 time to create 1 rle with old method : 0.0004112720489501953 time for calcul the mask position with numpy : 0.016741514205932617 nb_pixel_total : 831 time to create 1 rle with old method : 0.0010709762573242188 time for calcul the mask position with numpy : 0.016141653060913086 nb_pixel_total : 353 time to create 1 rle with old method : 0.0004630088806152344 time for calcul the mask position with numpy : 0.01639270782470703 nb_pixel_total : 409 time to create 1 rle with old method : 0.0007865428924560547 time for calcul the mask position with numpy : 0.01691746711730957 nb_pixel_total : 1898 time to create 1 rle with old method : 0.0033190250396728516 time for calcul the mask position with numpy : 0.018103599548339844 nb_pixel_total : 722 time to create 1 rle with old method : 0.0008733272552490234 time for calcul the mask position with numpy : 0.015669822692871094 nb_pixel_total : 27 time to create 1 rle with old method : 0.0001342296600341797 time for calcul the mask position with numpy : 0.015425920486450195 nb_pixel_total : 1279 time to create 1 rle with old method : 0.0014867782592773438 time for calcul the mask position with numpy : 0.015177488327026367 nb_pixel_total : 433 time to create 1 rle with old method : 0.0005185604095458984 time for calcul the mask position with numpy : 0.01521611213684082 nb_pixel_total : 1739 time to create 1 rle with old method : 0.0026633739471435547 time for calcul the mask position with numpy : 0.01587843894958496 nb_pixel_total : 506 time to create 1 rle with old method : 0.0006165504455566406 time for calcul the mask position with numpy : 0.01554250717163086 nb_pixel_total : 1794 time to create 1 rle with old method : 0.002166271209716797 time for calcul the mask position with numpy : 0.015548944473266602 nb_pixel_total : 917 time to create 1 rle with old method : 0.0011134147644042969 time for calcul the mask position with numpy : 0.015367984771728516 nb_pixel_total : 187 time to create 1 rle with old method : 0.0002505779266357422 time for calcul the mask position with numpy : 0.01542210578918457 nb_pixel_total : 2032 time to create 1 rle with old method : 0.002343416213989258 time for calcul the mask position with numpy : 0.0158233642578125 nb_pixel_total : 474 time to create 1 rle with old method : 0.0009257793426513672 time for calcul the mask position with numpy : 0.01508641242980957 nb_pixel_total : 4189 time to create 1 rle with old method : 0.004729270935058594 time for calcul the mask position with numpy : 0.015313863754272461 nb_pixel_total : 1789 time to create 1 rle with old method : 0.002058267593383789 time for calcul the mask position with numpy : 0.015880823135375977 nb_pixel_total : 181 time to create 1 rle with old method : 0.0002346038818359375 time for calcul the mask position with numpy : 0.015300750732421875 nb_pixel_total : 271 time to create 1 rle with old method : 0.0003631114959716797 time for calcul the mask position with numpy : 0.015831947326660156 nb_pixel_total : 564 time to create 1 rle with old method : 0.0006721019744873047 time for calcul the mask position with numpy : 0.015823841094970703 nb_pixel_total : 1999 time to create 1 rle with old method : 0.0025403499603271484 time for calcul the mask position with numpy : 0.016002893447875977 nb_pixel_total : 1075 time to create 1 rle with old method : 0.0012483596801757812 time for calcul the mask position with numpy : 0.015629053115844727 nb_pixel_total : 140 time to create 1 rle with old method : 0.00019812583923339844 time for calcul the mask position with numpy : 0.018854618072509766 nb_pixel_total : 692 time to create 1 rle with old method : 0.0010099411010742188 time for calcul the mask position with numpy : 0.015796184539794922 nb_pixel_total : 1468 time to create 1 rle with old method : 0.0021638870239257812 time for calcul the mask position with numpy : 0.0176999568939209 nb_pixel_total : 51 time to create 1 rle with old method : 0.00010895729064941406 time for calcul the mask position with numpy : 0.015615701675415039 nb_pixel_total : 429 time to create 1 rle with old method : 0.0005218982696533203 time for calcul the mask position with numpy : 0.016376972198486328 nb_pixel_total : 55 time to create 1 rle with old method : 0.00012493133544921875 time for calcul the mask position with numpy : 0.015529632568359375 nb_pixel_total : 366 time to create 1 rle with old method : 0.0004620552062988281 time for calcul the mask position with numpy : 0.015153169631958008 nb_pixel_total : 520 time to create 1 rle with old method : 0.0006606578826904297 time for calcul the mask position with numpy : 0.015422582626342773 nb_pixel_total : 425 time to create 1 rle with old method : 0.0005192756652832031 time for calcul the mask position with numpy : 0.015190362930297852 nb_pixel_total : 1114 time to create 1 rle with old method : 0.0013394355773925781 time for calcul the mask position with numpy : 0.015231847763061523 nb_pixel_total : 572 time to create 1 rle with old method : 0.0006895065307617188 time for calcul the mask position with numpy : 0.015494108200073242 nb_pixel_total : 1090 time to create 1 rle with old method : 0.0012874603271484375 time for calcul the mask position with numpy : 0.01606893539428711 nb_pixel_total : 2003 time to create 1 rle with old method : 0.0028395652770996094 time for calcul the mask position with numpy : 0.01564502716064453 nb_pixel_total : 3157 time to create 1 rle with old method : 0.0036699771881103516 time for calcul the mask position with numpy : 0.015438318252563477 nb_pixel_total : 287 time to create 1 rle with old method : 0.0005865097045898438 time for calcul the mask position with numpy : 0.015178680419921875 nb_pixel_total : 1561 time to create 1 rle with old method : 0.0018146038055419922 time for calcul the mask position with numpy : 0.01511836051940918 nb_pixel_total : 323 time to create 1 rle with old method : 0.0004031658172607422 time for calcul the mask position with numpy : 0.015513896942138672 nb_pixel_total : 252 time to create 1 rle with old method : 0.00048279762268066406 time for calcul the mask position with numpy : 0.015651702880859375 nb_pixel_total : 292 time to create 1 rle with old method : 0.00039196014404296875 time for calcul the mask position with numpy : 0.01685500144958496 nb_pixel_total : 808 time to create 1 rle with old method : 0.0009615421295166016 time for calcul the mask position with numpy : 0.015453815460205078 nb_pixel_total : 587 time to create 1 rle with old method : 0.0007283687591552734 time for calcul the mask position with numpy : 0.015099525451660156 nb_pixel_total : 347 time to create 1 rle with old method : 0.00040793418884277344 time for calcul the mask position with numpy : 0.015598297119140625 nb_pixel_total : 532 time to create 1 rle with old method : 0.0008258819580078125 time for calcul the mask position with numpy : 0.019058704376220703 nb_pixel_total : 190 time to create 1 rle with old method : 0.00028133392333984375 time for calcul the mask position with numpy : 0.015608072280883789 nb_pixel_total : 104 time to create 1 rle with old method : 0.00016069412231445312 time for calcul the mask position with numpy : 0.015468358993530273 nb_pixel_total : 112 time to create 1 rle with old method : 0.0001914501190185547 time for calcul the mask position with numpy : 0.016216039657592773 nb_pixel_total : 421 time to create 1 rle with old method : 0.0007686614990234375 time for calcul the mask position with numpy : 0.020023107528686523 nb_pixel_total : 3 time to create 1 rle with old method : 4.982948303222656e-05 time for calcul the mask position with numpy : 0.02049732208251953 nb_pixel_total : 178 time to create 1 rle with old method : 0.00033593177795410156 time for calcul the mask position with numpy : 0.020472049713134766 nb_pixel_total : 482 time to create 1 rle with old method : 0.0009050369262695312 time for calcul the mask position with numpy : 0.01970672607421875 nb_pixel_total : 402 time to create 1 rle with old method : 0.0007615089416503906 time for calcul the mask position with numpy : 0.0198671817779541 nb_pixel_total : 256 time to create 1 rle with old method : 0.0005524158477783203 time for calcul the mask position with numpy : 0.02010488510131836 nb_pixel_total : 185 time to create 1 rle with old method : 0.0004329681396484375 time for calcul the mask position with numpy : 0.01894545555114746 nb_pixel_total : 240 time to create 1 rle with old method : 0.00029659271240234375 time for calcul the mask position with numpy : 0.01575493812561035 nb_pixel_total : 40 time to create 1 rle with old method : 0.00011968612670898438 time for calcul the mask position with numpy : 0.015839338302612305 nb_pixel_total : 9 time to create 1 rle with old method : 4.38690185546875e-05 time for calcul the mask position with numpy : 0.015728473663330078 nb_pixel_total : 283 time to create 1 rle with old method : 0.0003685951232910156 time for calcul the mask position with numpy : 0.01570582389831543 nb_pixel_total : 142 time to create 1 rle with old method : 0.00032258033752441406 time for calcul the mask position with numpy : 0.015804290771484375 nb_pixel_total : 7147 time to create 1 rle with old method : 0.008345842361450195 time for calcul the mask position with numpy : 0.01570725440979004 nb_pixel_total : 241 time to create 1 rle with old method : 0.0003211498260498047 time for calcul the mask position with numpy : 0.01606130599975586 nb_pixel_total : 29 time to create 1 rle with old method : 0.000125885009765625 time for calcul the mask position with numpy : 0.01604294776916504 nb_pixel_total : 108 time to create 1 rle with old method : 0.0001583099365234375 time for calcul the mask position with numpy : 0.017233848571777344 nb_pixel_total : 1396 time to create 1 rle with old method : 0.0024628639221191406 time for calcul the mask position with numpy : 0.01924586296081543 nb_pixel_total : 1091 time to create 1 rle with old method : 0.0012917518615722656 time for calcul the mask position with numpy : 0.015856266021728516 nb_pixel_total : 84 time to create 1 rle with old method : 0.00016045570373535156 time for calcul the mask position with numpy : 0.01645946502685547 nb_pixel_total : 1185 time to create 1 rle with old method : 0.001399993896484375 time for calcul the mask position with numpy : 0.0159604549407959 nb_pixel_total : 2390 time to create 1 rle with old method : 0.0034637451171875 time for calcul the mask position with numpy : 0.017304658889770508 nb_pixel_total : 914 time to create 1 rle with old method : 0.0014133453369140625 time for calcul the mask position with numpy : 0.016341447830200195 nb_pixel_total : 826 time to create 1 rle with old method : 0.001007080078125 time for calcul the mask position with numpy : 0.0166168212890625 nb_pixel_total : 492 time to create 1 rle with old method : 0.0006473064422607422 time for calcul the mask position with numpy : 0.016101360321044922 nb_pixel_total : 8 time to create 1 rle with old method : 5.078315734863281e-05 time for calcul the mask position with numpy : 0.016154766082763672 nb_pixel_total : 436 time to create 1 rle with old method : 0.0005970001220703125 time for calcul the mask position with numpy : 0.016794204711914062 nb_pixel_total : 139 time to create 1 rle with old method : 0.0002079010009765625 time for calcul the mask position with numpy : 0.01586174964904785 nb_pixel_total : 457 time to create 1 rle with old method : 0.0006337165832519531 time for calcul the mask position with numpy : 0.016228437423706055 nb_pixel_total : 9 time to create 1 rle with old method : 9.918212890625e-05 time for calcul the mask position with numpy : 0.015592098236083984 nb_pixel_total : 523 time to create 1 rle with old method : 0.0006158351898193359 time for calcul the mask position with numpy : 0.015654325485229492 nb_pixel_total : 213 time to create 1 rle with old method : 0.0003120899200439453 time for calcul the mask position with numpy : 0.015623807907104492 nb_pixel_total : 1641 time to create 1 rle with old method : 0.0019464492797851562 time for calcul the mask position with numpy : 0.015642166137695312 nb_pixel_total : 561 time to create 1 rle with old method : 0.0006806850433349609 time for calcul the mask position with numpy : 0.015394449234008789 nb_pixel_total : 807 time to create 1 rle with old method : 0.001024007797241211 time for calcul the mask position with numpy : 0.0161285400390625 nb_pixel_total : 361 time to create 1 rle with old method : 0.00048232078552246094 time for calcul the mask position with numpy : 0.01537013053894043 nb_pixel_total : 240 time to create 1 rle with old method : 0.00031185150146484375 time for calcul the mask position with numpy : 0.015639066696166992 nb_pixel_total : 312 time to create 1 rle with old method : 0.00040841102600097656 time for calcul the mask position with numpy : 0.015778303146362305 nb_pixel_total : 730 time to create 1 rle with old method : 0.0008757114410400391 time for calcul the mask position with numpy : 0.017195940017700195 nb_pixel_total : 940 time to create 1 rle with old method : 0.0010933876037597656 time for calcul the mask position with numpy : 0.015445470809936523 nb_pixel_total : 12 time to create 1 rle with old method : 7.414817810058594e-05 time for calcul the mask position with numpy : 0.015651941299438477 nb_pixel_total : 2540 time to create 1 rle with old method : 0.00323486328125 time for calcul the mask position with numpy : 0.01592230796813965 nb_pixel_total : 276 time to create 1 rle with old method : 0.0003554821014404297 time for calcul the mask position with numpy : 0.015436410903930664 nb_pixel_total : 144 time to create 1 rle with old method : 0.00021696090698242188 time for calcul the mask position with numpy : 0.015453338623046875 nb_pixel_total : 1127 time to create 1 rle with old method : 0.0013382434844970703 time for calcul the mask position with numpy : 0.015466928482055664 nb_pixel_total : 16 time to create 1 rle with old method : 8.916854858398438e-05 time for calcul the mask position with numpy : 0.015227794647216797 nb_pixel_total : 606 time to create 1 rle with old method : 0.0007190704345703125 time for calcul the mask position with numpy : 0.015178680419921875 nb_pixel_total : 246 time to create 1 rle with old method : 0.0003376007080078125 time for calcul the mask position with numpy : 0.01517796516418457 nb_pixel_total : 350 time to create 1 rle with old method : 0.0004150867462158203 time for calcul the mask position with numpy : 0.01536703109741211 nb_pixel_total : 289 time to create 1 rle with old method : 0.0004608631134033203 time for calcul the mask position with numpy : 0.016568422317504883 nb_pixel_total : 1435 time to create 1 rle with old method : 0.0016894340515136719 time for calcul the mask position with numpy : 0.01526641845703125 nb_pixel_total : 794 time to create 1 rle with old method : 0.0009398460388183594 time for calcul the mask position with numpy : 0.015109539031982422 nb_pixel_total : 306 time to create 1 rle with old method : 0.00038909912109375 time for calcul the mask position with numpy : 0.015443801879882812 nb_pixel_total : 4005 time to create 1 rle with old method : 0.004618167877197266 time for calcul the mask position with numpy : 0.01534724235534668 nb_pixel_total : 1518 time to create 1 rle with old method : 0.001844167709350586 time for calcul the mask position with numpy : 0.016416311264038086 nb_pixel_total : 877 time to create 1 rle with old method : 0.0012862682342529297 time for calcul the mask position with numpy : 0.01634502410888672 nb_pixel_total : 1936 time to create 1 rle with old method : 0.002331972122192383 time for calcul the mask position with numpy : 0.015271425247192383 nb_pixel_total : 176 time to create 1 rle with old method : 0.0002429485321044922 time for calcul the mask position with numpy : 0.015574932098388672 nb_pixel_total : 1587 time to create 1 rle with old method : 0.001905679702758789 time for calcul the mask position with numpy : 0.015439510345458984 nb_pixel_total : 333 time to create 1 rle with old method : 0.0004265308380126953 time for calcul the mask position with numpy : 0.015431880950927734 nb_pixel_total : 322 time to create 1 rle with old method : 0.0005702972412109375 time for calcul the mask position with numpy : 0.01523733139038086 nb_pixel_total : 996 time to create 1 rle with old method : 0.0011889934539794922 time for calcul the mask position with numpy : 0.015431880950927734 nb_pixel_total : 650 time to create 1 rle with old method : 0.0007710456848144531 time for calcul the mask position with numpy : 0.017351150512695312 nb_pixel_total : 1374 time to create 1 rle with old method : 0.0017626285552978516 time for calcul the mask position with numpy : 0.015395164489746094 nb_pixel_total : 2061 time to create 1 rle with old method : 0.0024590492248535156 time for calcul the mask position with numpy : 0.0152740478515625 nb_pixel_total : 2406 time to create 1 rle with old method : 0.002752065658569336 time for calcul the mask position with numpy : 0.015381336212158203 nb_pixel_total : 160 time to create 1 rle with old method : 0.0002751350402832031 time for calcul the mask position with numpy : 0.015376091003417969 nb_pixel_total : 1949 time to create 1 rle with old method : 0.002233266830444336 time for calcul the mask position with numpy : 0.015235662460327148 nb_pixel_total : 188 time to create 1 rle with old method : 0.0002579689025878906 time for calcul the mask position with numpy : 0.01523733139038086 nb_pixel_total : 256 time to create 1 rle with old method : 0.0003185272216796875 time for calcul the mask position with numpy : 0.015288591384887695 nb_pixel_total : 1436 time to create 1 rle with old method : 0.0017025470733642578 time for calcul the mask position with numpy : 0.015162229537963867 nb_pixel_total : 337 time to create 1 rle with old method : 0.0004131793975830078 time for calcul the mask position with numpy : 0.015101194381713867 nb_pixel_total : 707 time to create 1 rle with old method : 0.0008525848388671875 time for calcul the mask position with numpy : 0.015178442001342773 nb_pixel_total : 68 time to create 1 rle with old method : 9.703636169433594e-05 time for calcul the mask position with numpy : 0.015302419662475586 nb_pixel_total : 786 time to create 1 rle with old method : 0.0009260177612304688 time for calcul the mask position with numpy : 0.015300273895263672 nb_pixel_total : 273 time to create 1 rle with old method : 0.00035953521728515625 time for calcul the mask position with numpy : 0.015139102935791016 nb_pixel_total : 1099 time to create 1 rle with old method : 0.0012996196746826172 time for calcul the mask position with numpy : 0.015108346939086914 nb_pixel_total : 275 time to create 1 rle with old method : 0.00034618377685546875 time for calcul the mask position with numpy : 0.015071630477905273 nb_pixel_total : 542 time to create 1 rle with old method : 0.0006740093231201172 time for calcul the mask position with numpy : 0.01518559455871582 nb_pixel_total : 28 time to create 1 rle with old method : 6.985664367675781e-05 time for calcul the mask position with numpy : 0.01515650749206543 nb_pixel_total : 175 time to create 1 rle with old method : 0.0002224445343017578 time for calcul the mask position with numpy : 0.015794754028320312 nb_pixel_total : 138 time to create 1 rle with old method : 0.00020503997802734375 time for calcul the mask position with numpy : 0.015108108520507812 nb_pixel_total : 256 time to create 1 rle with old method : 0.00033473968505859375 time for calcul the mask position with numpy : 0.015230655670166016 nb_pixel_total : 482 time to create 1 rle with old method : 0.0006582736968994141 time for calcul the mask position with numpy : 0.015155315399169922 nb_pixel_total : 106 time to create 1 rle with old method : 0.00013828277587890625 time for calcul the mask position with numpy : 0.015385627746582031 nb_pixel_total : 352 time to create 1 rle with old method : 0.0004649162292480469 time for calcul the mask position with numpy : 0.015471935272216797 nb_pixel_total : 557 time to create 1 rle with old method : 0.0006883144378662109 time for calcul the mask position with numpy : 0.015618324279785156 nb_pixel_total : 253 time to create 1 rle with old method : 0.00030422210693359375 time for calcul the mask position with numpy : 0.015114784240722656 nb_pixel_total : 181 time to create 1 rle with old method : 0.0002315044403076172 time for calcul the mask position with numpy : 0.017970800399780273 nb_pixel_total : 256 time to create 1 rle with old method : 0.0003235340118408203 time for calcul the mask position with numpy : 0.0149688720703125 nb_pixel_total : 289 time to create 1 rle with old method : 0.0003650188446044922 time for calcul the mask position with numpy : 0.015000343322753906 nb_pixel_total : 2123 time to create 1 rle with old method : 0.002486705780029297 time for calcul the mask position with numpy : 0.015009880065917969 nb_pixel_total : 1167 time to create 1 rle with old method : 0.0013575553894042969 time for calcul the mask position with numpy : 0.014940738677978516 nb_pixel_total : 180 time to create 1 rle with old method : 0.00026607513427734375 time for calcul the mask position with numpy : 0.015445709228515625 nb_pixel_total : 1433 time to create 1 rle with old method : 0.0016973018646240234 time for calcul the mask position with numpy : 0.015129566192626953 nb_pixel_total : 653 time to create 1 rle with old method : 0.0007967948913574219 time for calcul the mask position with numpy : 0.01517486572265625 nb_pixel_total : 280 time to create 1 rle with old method : 0.0004825592041015625 time for calcul the mask position with numpy : 0.01528024673461914 nb_pixel_total : 1329 time to create 1 rle with old method : 0.00152587890625 time for calcul the mask position with numpy : 0.015194177627563477 nb_pixel_total : 3542 time to create 1 rle with old method : 0.004065990447998047 time for calcul the mask position with numpy : 0.015088558197021484 nb_pixel_total : 169 time to create 1 rle with old method : 0.00022745132446289062 time for calcul the mask position with numpy : 0.015051603317260742 nb_pixel_total : 468 time to create 1 rle with old method : 0.0006029605865478516 time for calcul the mask position with numpy : 0.015075445175170898 nb_pixel_total : 689 time to create 1 rle with old method : 0.0008122920989990234 time for calcul the mask position with numpy : 0.014973878860473633 nb_pixel_total : 587 time to create 1 rle with old method : 0.0007145404815673828 time for calcul the mask position with numpy : 0.014930248260498047 nb_pixel_total : 715 time to create 1 rle with old method : 0.0008287429809570312 time for calcul the mask position with numpy : 0.014744997024536133 nb_pixel_total : 690 time to create 1 rle with old method : 0.0008358955383300781 time for calcul the mask position with numpy : 0.01496124267578125 nb_pixel_total : 91 time to create 1 rle with old method : 0.0001289844512939453 time for calcul the mask position with numpy : 0.015088558197021484 nb_pixel_total : 6 time to create 1 rle with old method : 2.4557113647460938e-05 time for calcul the mask position with numpy : 0.015317440032958984 nb_pixel_total : 973 time to create 1 rle with old method : 0.0012149810791015625 time for calcul the mask position with numpy : 0.015206098556518555 nb_pixel_total : 1635 time to create 1 rle with old method : 0.0019779205322265625 time for calcul the mask position with numpy : 0.015021800994873047 nb_pixel_total : 245 time to create 1 rle with old method : 0.0003414154052734375 time for calcul the mask position with numpy : 0.014991044998168945 nb_pixel_total : 630 time to create 1 rle with old method : 0.0007476806640625 time for calcul the mask position with numpy : 0.01499032974243164 nb_pixel_total : 1830 time to create 1 rle with old method : 0.002161741256713867 time for calcul the mask position with numpy : 0.015050888061523438 nb_pixel_total : 178 time to create 1 rle with old method : 0.00030040740966796875 time for calcul the mask position with numpy : 0.015049457550048828 nb_pixel_total : 1211 time to create 1 rle with old method : 0.0015304088592529297 time for calcul the mask position with numpy : 0.015947818756103516 nb_pixel_total : 442 time to create 1 rle with old method : 0.00054931640625 time for calcul the mask position with numpy : 0.014848947525024414 nb_pixel_total : 322 time to create 1 rle with old method : 0.00038909912109375 time for calcul the mask position with numpy : 0.01483297348022461 nb_pixel_total : 1928 time to create 1 rle with old method : 0.0022246837615966797 time for calcul the mask position with numpy : 0.014872550964355469 nb_pixel_total : 895 time to create 1 rle with old method : 0.000985860824584961 time for calcul the mask position with numpy : 0.015039682388305664 nb_pixel_total : 245 time to create 1 rle with old method : 0.00030612945556640625 time for calcul the mask position with numpy : 0.015177011489868164 nb_pixel_total : 888 time to create 1 rle with old method : 0.0010526180267333984 time for calcul the mask position with numpy : 0.015536069869995117 nb_pixel_total : 307 time to create 1 rle with old method : 0.00038886070251464844 time for calcul the mask position with numpy : 0.015269994735717773 nb_pixel_total : 487 time to create 1 rle with old method : 0.0006108283996582031 time for calcul the mask position with numpy : 0.015230417251586914 nb_pixel_total : 356 time to create 1 rle with old method : 0.0005123615264892578 time for calcul the mask position with numpy : 0.015038490295410156 nb_pixel_total : 220 time to create 1 rle with old method : 0.0002684593200683594 time for calcul the mask position with numpy : 0.014996051788330078 nb_pixel_total : 2947 time to create 1 rle with old method : 0.0033636093139648438 time for calcul the mask position with numpy : 0.014898300170898438 nb_pixel_total : 221 time to create 1 rle with old method : 0.0003476142883300781 time for calcul the mask position with numpy : 0.015096426010131836 nb_pixel_total : 16 time to create 1 rle with old method : 5.078315734863281e-05 time for calcul the mask position with numpy : 0.015059947967529297 nb_pixel_total : 1271 time to create 1 rle with old method : 0.0014443397521972656 time for calcul the mask position with numpy : 0.015326499938964844 nb_pixel_total : 122 time to create 1 rle with old method : 0.0001811981201171875 time for calcul the mask position with numpy : 0.015458345413208008 nb_pixel_total : 383 time to create 1 rle with old method : 0.000476837158203125 time for calcul the mask position with numpy : 0.015458106994628906 nb_pixel_total : 300 time to create 1 rle with old method : 0.00038623809814453125 time for calcul the mask position with numpy : 0.015309333801269531 nb_pixel_total : 133 time to create 1 rle with old method : 0.0001895427703857422 time for calcul the mask position with numpy : 0.01529693603515625 nb_pixel_total : 227 time to create 1 rle with old method : 0.00030922889709472656 time for calcul the mask position with numpy : 0.015301942825317383 nb_pixel_total : 617 time to create 1 rle with old method : 0.00075531005859375 time for calcul the mask position with numpy : 0.015434741973876953 nb_pixel_total : 477 time to create 1 rle with old method : 0.0006892681121826172 time for calcul the mask position with numpy : 0.01526188850402832 nb_pixel_total : 278 time to create 1 rle with old method : 0.0003807544708251953 time for calcul the mask position with numpy : 0.01680779457092285 nb_pixel_total : 953 time to create 1 rle with old method : 0.0011091232299804688 time for calcul the mask position with numpy : 0.015162944793701172 nb_pixel_total : 46 time to create 1 rle with old method : 7.605552673339844e-05 time for calcul the mask position with numpy : 0.015004158020019531 nb_pixel_total : 678 time to create 1 rle with old method : 0.0008034706115722656 time for calcul the mask position with numpy : 0.016807079315185547 nb_pixel_total : 1440 time to create 1 rle with old method : 0.0016665458679199219 time for calcul the mask position with numpy : 0.014940500259399414 nb_pixel_total : 166 time to create 1 rle with old method : 0.00019550323486328125 time for calcul the mask position with numpy : 0.014827489852905273 nb_pixel_total : 96 time to create 1 rle with old method : 0.00014972686767578125 time for calcul the mask position with numpy : 0.015264272689819336 nb_pixel_total : 863 time to create 1 rle with old method : 0.001035928726196289 time for calcul the mask position with numpy : 0.01516103744506836 nb_pixel_total : 564 time to create 1 rle with old method : 0.000675201416015625 time for calcul the mask position with numpy : 0.015385150909423828 nb_pixel_total : 295 time to create 1 rle with old method : 0.00045108795166015625 time for calcul the mask position with numpy : 0.01685190200805664 nb_pixel_total : 9180 time to create 1 rle with old method : 0.010837793350219727 time for calcul the mask position with numpy : 0.015381336212158203 nb_pixel_total : 391 time to create 1 rle with old method : 0.0004591941833496094 time for calcul the mask position with numpy : 0.015355825424194336 nb_pixel_total : 227 time to create 1 rle with old method : 0.00030684471130371094 time for calcul the mask position with numpy : 0.015317201614379883 nb_pixel_total : 1739 time to create 1 rle with old method : 0.002059459686279297 time for calcul the mask position with numpy : 0.015259265899658203 nb_pixel_total : 26 time to create 1 rle with old method : 7.915496826171875e-05 time for calcul the mask position with numpy : 0.015253067016601562 nb_pixel_total : 813 time to create 1 rle with old method : 0.0009706020355224609 time for calcul the mask position with numpy : 0.015229225158691406 nb_pixel_total : 1113 time to create 1 rle with old method : 0.00127410888671875 time for calcul the mask position with numpy : 0.015065670013427734 nb_pixel_total : 519 time to create 1 rle with old method : 0.0006544589996337891 time for calcul the mask position with numpy : 0.015112638473510742 nb_pixel_total : 3969 time to create 1 rle with old method : 0.004500627517700195 time for calcul the mask position with numpy : 0.015066146850585938 nb_pixel_total : 109 time to create 1 rle with old method : 0.00015473365783691406 time for calcul the mask position with numpy : 0.015232563018798828 nb_pixel_total : 350 time to create 1 rle with old method : 0.00042700767517089844 time for calcul the mask position with numpy : 0.015380382537841797 nb_pixel_total : 196 time to create 1 rle with old method : 0.0002574920654296875 time for calcul the mask position with numpy : 0.015128850936889648 nb_pixel_total : 1246 time to create 1 rle with old method : 0.0014684200286865234 time for calcul the mask position with numpy : 0.015291452407836914 nb_pixel_total : 594 time to create 1 rle with old method : 0.0007314682006835938 time for calcul the mask position with numpy : 0.015255451202392578 nb_pixel_total : 440 time to create 1 rle with old method : 0.0006368160247802734 time for calcul the mask position with numpy : 0.015302658081054688 nb_pixel_total : 249 time to create 1 rle with old method : 0.00033092498779296875 time for calcul the mask position with numpy : 0.015255451202392578 nb_pixel_total : 669 time to create 1 rle with old method : 0.0007944107055664062 time for calcul the mask position with numpy : 0.017513751983642578 nb_pixel_total : 21764 time to create 1 rle with old method : 0.02417469024658203 time for calcul the mask position with numpy : 0.015329122543334961 nb_pixel_total : 485 time to create 1 rle with old method : 0.0005674362182617188 time for calcul the mask position with numpy : 0.015639781951904297 nb_pixel_total : 2865 time to create 1 rle with old method : 0.003657817840576172 time for calcul the mask position with numpy : 0.015391349792480469 nb_pixel_total : 367 time to create 1 rle with old method : 0.000461578369140625 time for calcul the mask position with numpy : 0.015306711196899414 nb_pixel_total : 19 time to create 1 rle with old method : 5.5789947509765625e-05 time for calcul the mask position with numpy : 0.015342235565185547 nb_pixel_total : 167 time to create 1 rle with old method : 0.0002124309539794922 time for calcul the mask position with numpy : 0.015274286270141602 nb_pixel_total : 149 time to create 1 rle with old method : 0.00022339820861816406 time for calcul the mask position with numpy : 0.015300989151000977 nb_pixel_total : 400 time to create 1 rle with old method : 0.0004863739013671875 time for calcul the mask position with numpy : 0.015728473663330078 nb_pixel_total : 181 time to create 1 rle with old method : 0.0002574920654296875 create new chi : 4.4567787647247314 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.012792825698852539 batch 1 Loaded 228 chid ids of type : 4230 Number RLEs to save : 17136 TO DO : save crop sub photo not yet done ! save time : 1.0543925762176514 map_output_result : {1340965857: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1340965857] Looping around the photos to save general results len do output : 1 /1340965857.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2636547') ('4234', '21053699', '1340965857', None, None, None, None, None, '2636547') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.011720657348632812 save_final save missing photos in datou_result : time spend for datou_step_exec : 7.6852686405181885 time spend to save output : 0.012018680572509766 total time spend for step 4 : 7.697287321090698 step5:crop_condition Wed Mar 5 09:21:05 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4230 Loading chi in step crop for list_pids : 1 ! batch 1 Loaded 228 chid ids of type : 4230 begin to crop the class : papier param for this class : {'min_score': 0.6} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 128 About to insert : list_path_to_insert length 128 new photo from crops ! About to upload 128 photos upload in portfolio : 4869462 init cache_photo without model_param we have 128 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741162869_141569 we have uploaded 128 photos in the portfolio 4869462 time of upload the photos Elapsed time : 35.23920202255249 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.6} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 21 About to insert : list_path_to_insert length 21 new photo from crops ! About to upload 21 photos upload in portfolio : 4869462 init cache_photo without model_param we have 21 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741162904_141569 we have uploaded 21 photos in the portfolio 4869462 time of upload the photos Elapsed time : 7.678717851638794 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.6} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.6} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! About to upload 8 photos upload in portfolio : 4869462 init cache_photo without model_param we have 8 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741162912_141569 we have uploaded 8 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.720252513885498 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.6} filtre for class : autre hashtag_id of this class : 494826614 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! About to upload 8 photos upload in portfolio : 4869462 init cache_photo without model_param we have 8 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741162915_141569 we have uploaded 8 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.6750781536102295 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.6} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.6} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1340965857] Looping around the photos to save general results len do output : 165 /1340974092Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974094Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974096Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974097Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974098Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974099Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974100Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974101Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974102Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974103Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974104Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974105Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974106Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974107Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974108Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974109Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974110Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974111Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974113Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974114Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974115Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974116Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974117Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974118Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974119Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974120Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974121Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974122Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974123Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974124Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974125Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974126Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974127Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974128Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974129Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974135Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974138Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974140Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974142Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974144Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974146Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974148Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974150Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974152Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974154Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974156Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974158Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974160Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974161Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974162Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974163Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974164Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974165Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974166Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974168Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974169Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974170Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974171Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974172Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974173Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974174Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974175Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974176Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974178Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974180Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974181Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974182Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974183Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974184Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974185Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974186Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974187Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974188Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974189Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974190Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974191Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974192Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974193Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974194Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974195Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974196Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974197Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974199Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974202Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974203Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974204Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974205Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974206Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974207Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974208Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974209Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974210Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974211Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974212Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974213Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974214Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974215Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974216Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974217Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974218Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974219Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974220Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974221Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974222Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974223Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974224Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974225Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974226Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974227Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974228Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974229Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974230Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974231Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974232Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974233Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974234Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974235Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974236Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974237Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974238Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974239Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974240Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974241Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974243Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974244Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974245Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974246Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974247Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974282Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974285Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974288Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974291Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974293Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974295Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974297Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974299Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974301Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974303Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974305Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974307Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974309Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974311Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974314Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974316Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974318Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974320Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974322Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974324Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974326Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974337Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974339Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974341Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974343Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974345Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974347Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974349Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974351Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974381Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974384Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974387Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974390Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974393Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974396Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974399Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340974402Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2636547') ('4234', '21053699', '1340965857', None, None, None, None, None, '2636547') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 496 time used for this insertion : 0.030869007110595703 save_final save missing photos in datou_result : time spend for datou_step_exec : 53.721352338790894 time spend to save output : 0.037580013275146484 total time spend for step 5 : 53.75893235206604 step6:thcl Wed Mar 5 09:21:59 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! we are using the classfication for only one thcl 3237 time to import caffe and check if the image exist : 0.006081581115722656 time to convert the images to numpy array : 0.028405427932739258 time to import caffe and check if the image exist : 0.01457071304321289 time to convert the images to numpy array : 0.054850101470947266 time to import caffe and check if the image exist : 0.008512735366821289 time to convert the images to numpy array : 0.0612025260925293 time to import caffe and check if the image exist : 0.011115312576293945 time to convert the images to numpy array : 0.05875658988952637 time to import caffe and check if the image exist : 0.016065120697021484 time to convert the images to numpy array : 0.0552668571472168 time to import caffe and check if the image exist : 0.014372825622558594 time to convert the images to numpy array : 0.058545589447021484 time to import caffe and check if the image exist : 0.011584043502807617 time to convert the images to numpy array : 0.06334757804870605 time to import caffe and check if the image exist : 0.012362241744995117 time to convert the images to numpy array : 0.06548833847045898 time to import caffe and check if the image exist : 0.02494359016418457 time to convert the images to numpy array : 0.053305625915527344 time to import caffe and check if the image exist : 0.023745059967041016 time to convert the images to numpy array : 0.05566120147705078 total time to convert the images to numpy array : 0.3837590217590332 list photo_ids error: [] list photo_ids correct : [1340974345, 1340974347, 1340974349, 1340974351, 1340974381, 1340974384, 1340974387, 1340974390, 1340974393, 1340974396, 1340974399, 1340974402, 1340974129, 1340974135, 1340974138, 1340974140, 1340974142, 1340974144, 1340974146, 1340974148, 1340974150, 1340974152, 1340974154, 1340974156, 1340974158, 1340974160, 1340974161, 1340974162, 1340974163, 1340974238, 1340974239, 1340974240, 1340974241, 1340974243, 1340974244, 1340974245, 1340974246, 1340974247, 1340974282, 1340974285, 1340974288, 1340974291, 1340974293, 1340974295, 1340974297, 1340974299, 1340974221, 1340974222, 1340974223, 1340974224, 1340974225, 1340974226, 1340974227, 1340974228, 1340974229, 1340974230, 1340974231, 1340974232, 1340974233, 1340974234, 1340974235, 1340974236, 1340974237, 1340974301, 1340974303, 1340974305, 1340974307, 1340974309, 1340974311, 1340974314, 1340974316, 1340974318, 1340974320, 1340974322, 1340974324, 1340974326, 1340974337, 1340974339, 1340974341, 1340974343, 1340974184, 1340974185, 1340974186, 1340974187, 1340974188, 1340974189, 1340974190, 1340974191, 1340974192, 1340974193, 1340974194, 1340974195, 1340974196, 1340974197, 1340974199, 1340974202, 1340974203, 1340974204, 1340974205, 1340974206, 1340974207, 1340974208, 1340974209, 1340974210, 1340974211, 1340974212, 1340974213, 1340974214, 1340974215, 1340974216, 1340974217, 1340974218, 1340974219, 1340974220, 1340974111, 1340974113, 1340974114, 1340974115, 1340974116, 1340974117, 1340974118, 1340974119, 1340974120, 1340974121, 1340974122, 1340974123, 1340974124, 1340974125, 1340974126, 1340974127, 1340974128, 1340974164, 1340974165, 1340974166, 1340974168, 1340974169, 1340974170, 1340974171, 1340974172, 1340974173, 1340974174, 1340974175, 1340974176, 1340974178, 1340974180, 1340974181, 1340974182, 1340974183, 1340974092, 1340974094, 1340974096, 1340974097, 1340974098, 1340974099, 1340974100, 1340974101, 1340974102, 1340974103, 1340974104, 1340974105, 1340974106, 1340974107, 1340974108, 1340974109, 1340974110] number of photos to traite : 165 try to delete the photos incorrect in DB tagging for thcl : 3237 To do loadFromThcl(), then load ParamDescType : thcl3237 thcls : [{'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'}] thcl {'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'} Update svm_hashtag_type_desc : 5561 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) To loadFromThcl() : net_5561 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10372 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_rubbia_refus_2500 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_rubbia_refus_2500 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_rubbia_refus_2500 /data/models_weight/learn_rubbia_refus_2500/caffemodel size_local : 94358479 size in s3 : 94358479 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:16 caffemodel already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt size_local : 32544 size in s3 : 32544 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:15 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:09:52 mean.npy already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/synset_words.txt size_local : 334 size in s3 : 334 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:10:06 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt caffemodel_filename : /data/models_weight/learn_rubbia_refus_2500/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10153 max_wait_temp : 1 max_wait : 0 dict_keys(['pool5', 'prob']) time used to do the prepocess of the images : 0.799954891204834 time used to do the prediction : 0.6958284378051758 save descriptor for thcl : 3237 time to traite the descriptors : 1.0124728679656982 Catched exception ! Connect or reconnect ! storage_type for insertDescriptorsMulti : 3 To insert : 1340974345 To insert : 1340974347 To insert : 1340974349 To insert : 1340974351 To insert : 1340974381 To insert : 1340974384 To insert : 1340974387 To insert : 1340974390 To insert : 1340974393 To insert : 1340974396 To insert : 1340974399 To insert : 1340974402 To insert : 1340974129 To insert : 1340974135 To insert : 1340974138 To insert : 1340974140 To insert : 1340974142 To insert : 1340974144 To insert : 1340974146 To insert : 1340974148 To insert : 1340974150 To insert : 1340974152 To insert : 1340974154 To insert : 1340974156 To insert : 1340974158 To insert : 1340974160 To insert : 1340974161 To insert : 1340974162 To insert : 1340974163 To insert : 1340974238 To insert : 1340974239 To insert : 1340974240 To insert : 1340974241 To insert : 1340974243 To insert : 1340974244 To insert : 1340974245 To insert : 1340974246 To insert : 1340974247 To insert : 1340974282 To insert : 1340974285 To insert : 1340974288 To insert : 1340974291 To insert : 1340974293 To insert : 1340974295 To insert : 1340974297 To insert : 1340974299 To insert : 1340974221 To insert : 1340974222 To insert : 1340974223 To insert : 1340974224 To insert : 1340974225 To insert : 1340974226 To insert : 1340974227 To insert : 1340974228 To insert : 1340974229 To insert : 1340974230 To insert : 1340974231 To insert : 1340974232 To insert : 1340974233 To insert : 1340974234 To insert : 1340974235 To insert : 1340974236 To insert : 1340974237 To insert : 1340974301 To insert : 1340974303 To insert : 1340974305 To insert : 1340974307 To insert : 1340974309 To insert : 1340974311 To insert : 1340974314 To insert : 1340974316 To insert : 1340974318 To insert : 1340974320 To insert : 1340974322 To insert : 1340974324 To insert : 1340974326 To insert : 1340974337 To insert : 1340974339 To insert : 1340974341 To insert : 1340974343 To insert : 1340974184 To insert : 1340974185 To insert : 1340974186 To insert : 1340974187 To insert : 1340974188 To insert : 1340974189 To insert : 1340974190 To insert : 1340974191 To insert : 1340974192 To insert : 1340974193 To insert : 1340974194 To insert : 1340974195 To insert : 1340974196 To insert : 1340974197 To insert : 1340974199 To insert : 1340974202 To insert : 1340974203 To insert : 1340974204 To insert : 1340974205 To insert : 1340974206 To insert : 1340974207 To insert : 1340974208 To insert : 1340974209 To insert : 1340974210 To insert : 1340974211 To insert : 1340974212 To insert : 1340974213 To insert : 1340974214 To insert : 1340974215 To insert : 1340974216 To insert : 1340974217 To insert : 1340974218 To insert : 1340974219 To insert : 1340974220 To insert : 1340974111 To insert : 1340974113 To insert : 1340974114 To insert : 1340974115 To insert : 1340974116 To insert : 1340974117 To insert : 1340974118 To insert : 1340974119 To insert : 1340974120 To insert : 1340974121 To insert : 1340974122 To insert : 1340974123 To insert : 1340974124 To insert : 1340974125 To insert : 1340974126 To insert : 1340974127 To insert : 1340974128 To insert : 1340974164 To insert : 1340974165 To insert : 1340974166 To insert : 1340974168 To insert : 1340974169 To insert : 1340974170 To insert : 1340974171 To insert : 1340974172 To insert : 1340974173 To insert : 1340974174 To insert : 1340974175 To insert : 1340974176 To insert : 1340974178 To insert : 1340974180 To insert : 1340974181 To insert : 1340974182 To insert : 1340974183 To insert : 1340974092 To insert : 1340974094 To insert : 1340974096 To insert : 1340974097 To insert : 1340974098 To insert : 1340974099 To insert : 1340974100 To insert : 1340974101 To insert : 1340974102 To insert : 1340974103 To insert : 1340974104 To insert : 1340974105 To insert : 1340974106 To insert : 1340974107 To insert : 1340974108 To insert : 1340974109 To insert : 1340974110 time to insert the descriptors : 34.36873769760132 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos SAVE THCL : begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 244 time used for this insertion : 0.02148127555847168 save missing photos in datou_result : time spend for datou_step_exec : 41.3018753528595 time spend to save output : 0.06963205337524414 total time spend for step 6 : 41.37150740623474 step7:ventilate_hashtags_in_portfolio Wed Mar 5 09:22:40 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 21053699 get user id for portfolio 21053699 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21053699 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Papier','Tetrapak','mal_croppe','PEHD','pet_fonce','PET_clair','metal','PET_fonce','papier','environnement','pehd','Carton','refus','Film_plastique','pet_clair','flou','carton','autre')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21053699 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Papier','Tetrapak','mal_croppe','PEHD','pet_fonce','PET_clair','metal','PET_fonce','papier','environnement','pehd','Carton','refus','Film_plastique','pet_clair','flou','carton','autre')) AND mptpi.`min_score`=0.6 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21053699 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Papier','Tetrapak','mal_croppe','PEHD','pet_fonce','PET_clair','metal','PET_fonce','papier','environnement','pehd','Carton','refus','Film_plastique','pet_clair','flou','carton','autre')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21053699 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Papier','Tetrapak','mal_croppe','PEHD','pet_fonce','PET_clair','metal','PET_fonce','papier','environnement','pehd','Carton','refus','Film_plastique','pet_clair','flou','carton','autre')) AND mptpi.`min_score`=0.6 To do lien utilise dans velours : https://www.fotonower.com/velours/21054479,21054472,21054473,21054481,21054478,21054485,21054477,21054480,21054487,21054483,21054484,21054486,21054488?tags=papier,tetrapak,mal_croppe,pehd,pet_fonce,pet_clair,metal,environnement,carton,refus,film_plastique,flou,autre&datou_id_consolidate=4235&port_consolidate=21053699 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1340965857] Looping around the photos to save general results len do output : 1 /21053699. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2636547') ('4234', '21053699', '1340965857', None, None, None, None, None, '2636547') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 2 time used for this insertion : 0.012481212615966797 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.988469123840332 time spend to save output : 0.01298069953918457 total time spend for step 7 : 3.0014498233795166 step8:final Wed Mar 5 09:22:43 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1340965857: ('0.8611441436098178',)} new output for save of step final : {1340965857: ('0.8611441436098178',)} [1340965857] Looping around the photos to save general results len do output : 1 /1340965857.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2636547') ('4234', '21053699', '1340965857', None, None, None, None, None, '2636547') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.0172574520111084 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.10738420486450195 time spend to save output : 0.017441272735595703 total time spend for step 8 : 0.12482547760009766 step9:velours_tree Wed Mar 5 09:22:43 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure list_portfolios : 21054479,21054472,21054473,21054481,21054478,21054485,21054477,21054480,21054487,21054483,21054484,21054486,21054488 photo desc type : 5561 - Retrieving photos to tag... query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21054479 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21054472 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21054473 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21054481 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21054478 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21054485 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21054477 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21054480 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21054487 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21054483 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21054484 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21054486 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21054488 ORDER BY ph.size desc - Loading descriptors... Size : 2048 len(descriptors) : 165 Compute structured hierarchical clustering... ward : AgglomerativeClustering(n_clusters=6) ward.labels_ : [0 3 0 1 1 0 2 1 1 1 5 1 1 5 2 4 0 5 1 2 0 0 4 5 4 5 4 4 1 1 0 5 2 1 1 1 5 2 3 1 2 1 1 4 3 2 1 3 4 5 1 2 0 2 0 1 0 0 4 2 3 5 4 4 1 2 4 2 1 0 4 1 2 4 0 3 4 5 3 3 2 4 4 1 4 3 0 4 0 4 4 4 5 1 2 4 2 2 4 0 0 0 4 0 0 0 0 0 1 0 4 4 1 0 4 4 0 3 0 5 0 4 4 2 0 2 0 2 2 0 0 0 1 0 0 4 1 4 0 0 1 0 2 2 0 2 0 4 2 3 5 1 1 1 1 5 0 3 4 1 1 1 0 0 0] Elapsed time: 0.030901670455932617 graph_id used : 81527 - Beta version, working pretty good on 11-5-16 ! https://www.fotonower.com/velours/21054479,21054472,21054473,21054481,21054478,21054485,21054477,21054480,21054487,21054483,21054484,21054486,21054488?tags=papier,tetrapak,mal_croppe,pehd,pet_fonce,pet_clair,metal,environnement,carton,refus,film_plastique,flou,autre&datou_id_consolidate=4235&port_consolidate=21053699&tree_id=81527 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : velours_tree we use saveGeneral [1340965857] Looping around the photos to save general results len do output : 1 /21053699Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2636547') ('4234', '21053699', '1340965857', None, None, None, None, None, '2636547') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 2 time used for this insertion : 0.012183427810668945 save_final save missing photos in datou_result : time spend for datou_step_exec : 10.983607053756714 time spend to save output : 0.012633562088012695 total time spend for step 9 : 10.996240615844727 step10:send_mail_cod Wed Mar 5 09:22:54 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 complete output_args for input 2 complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Qualipapia_P21053699_05-03-2025_09_22_54.pdf 21054447 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette210544471741162974 21054440 imagette210544401741162975 21054441 imagette210544411741162975 21054457 imagette210544571741162975 21054446 imagette210544461741162975 21054467 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette210544671741162975 21054445 imagette210544451741162976 21054469 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette210544691741162976 21054465 imagette210544651741162977 21054466 imagette210544661741162977 21054468 imagette210544681741162977 21054470 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette210544701741162977 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=21053699 and hashtag_type = 4230 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/21054479,21054472,21054473,21054481,21054478,21054485,21054477,21054480,21054487,21054483,21054484,21054486,21054488?tags=papier,tetrapak,mal_croppe,pehd,pet_fonce,pet_clair,metal,environnement,carton,refus,film_plastique,flou,autre&datou_id_consolidate=4235&port_consolidate=21053699&tree_id=81527 your option no_mail is active, we will not send the real mail to your client args[1340965857] : ((1340965857, -7.225758406722352, 492609224), (1340965857, 0.5063696996966589, 2107752395), '0.8611441436098178') We are sending mail with results at cod@fotonower.com refus_total : 0.8611441436098178 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21053699 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21053699_05-03-2025_09_22_54.pdf results_Qualipapia_P21053699_05-03-2025_09_22_54.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21053699_05-03-2025_09_22_54.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4234','21053699','results_Qualipapia_P21053699_05-03-2025_09_22_54.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21053699_05-03-2025_09_22_54.pdf','pdf','','0.12','0.8611441436098178') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1340965857] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2636547') ('4234', '21053699', '1340965857', None, None, None, None, None, '2636547') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 1 time used for this insertion : 0.01173710823059082 save_final save missing photos in datou_result : time spend for datou_step_exec : 3.4265503883361816 time spend to save output : 0.011977434158325195 total time spend for step 10 : 3.438527822494507 step11:split_time_score Wed Mar 5 09:22:58 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 3442, 'mtr_user_id': 31, 'name': 'classifieur_2camions_valcor_021122_v1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'deux_camions,camion_droite,camion_gauche,pas_de_camion', 'svm_portfolios_learning': '7659379,7659034,7657685,7657114', 'photo_hashtag_type': 4458, 'photo_desc_type': 5723, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107760533,2107760534,2107760535,2107760536'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('20', 1),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 05032025 21053699 Nombre de photos uploadées : 1 / 23040 (0%) 05032025 21053699 Nombre de photos taguées (types de déchets): 0 / 1 (0%) 05032025 21053699 Nombre de photos taguées (volume) : 0 / 1 (0%) elapsed_time : load_data_split_time_score 2.86102294921875e-06 elapsed_time : order_list_meta_photo_and_scores 8.344650268554688e-06 ? elapsed_time : fill_and_build_computed_from_old_data 0.00017070770263671875 elapsed_time : insert_dashboard_record_day_entry 0.022705078125 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.811510457039575 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21036990_05-03-2025_02_19_40.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21036990 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21036990 AND mptpi.`type`=4230 To do Qualite : 0.8128125516983411 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21036991_05-03-2025_02_06_22.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21036991 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21036991 AND mptpi.`type`=4230 To do Qualite : 0.8128500134519651 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21039249_05-03-2025_03_18_04.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21039249 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21039249 AND mptpi.`type`=4230 To do Qualite : 0.7928432280282436 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21041867_05-03-2025_04_18_41.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21041867 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21041867 AND mptpi.`type`=4230 To do Qualite : 0.8058749043180025 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21043683_05-03-2025_05_05_35.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21043683 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21043683 AND mptpi.`type`=4230 To do Qualite : 0.8214959577213765 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21046296_05-03-2025_06_18_21.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21046296 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21046296 AND mptpi.`type`=4230 To do Qualite : 0.7920257507424305 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21048819_05-03-2025_07_03_32.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21048819 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21048819 AND mptpi.`type`=4230 To do Qualite : 0.8611441436098178 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21053699_05-03-2025_09_22_54.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21053699 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21053699 AND mptpi.`type`=4230 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'05032025': {'nb_upload': 1, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1340965857] Looping around the photos to save general results len do output : 1 /21053699Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2636547') ('4234', '21053699', '1340965857', None, None, None, None, None, '2636547') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 2 time used for this insertion : 0.011587858200073242 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.6695895195007324 time spend to save output : 0.011732339859008789 total time spend for step 11 : 0.6813218593597412 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 1 len(input) + len(total_photo_id_missing) : 1 set_done_treatment 35.82user 22.25system 2:32.95elapsed 37%CPU (0avgtext+0avgdata 2440992maxresident)k 775576inputs+11104outputs (1102major+2936572minor)pagefaults 0swaps