python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 4234 -l 5 ' -s datou_current_4234 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3215133 load datou : 4234 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load THCL from format json or kwargs add thcl : 3237 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig add pdt : 5561 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4234, datou_cur_ids : ['2634225'] with mtr_portfolio_ids : ['21027671'] and first list_photo_ids : [] new path : /proc/3215133/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, brightness, blur_detection, rle_unique_nms_with_priority, crop_condition, thcl, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , BFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 1 ; length of list_pids : 1 ; length of list_args : 1 time to download the photos : 0.5469000339508057 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Wed Mar 5 04:20:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10593 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-03-05 04:20:33.842141: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-03-05 04:20:33.871141: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-03-05 04:20:33.873288: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7efea4000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-03-05 04:20:33.873341: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-03-05 04:20:33.877768: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-03-05 04:20:34.125554: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x4be0bd0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-03-05 04:20:34.125611: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-03-05 04:20:34.126936: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-05 04:20:34.127391: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-05 04:20:34.130454: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-05 04:20:34.134384: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-05 04:20:34.134712: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-05 04:20:34.139309: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-05 04:20:34.140867: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-05 04:20:34.149681: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-05 04:20:34.151566: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-05 04:20:34.151668: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-05 04:20:34.152616: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-05 04:20:34.152636: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-05 04:20:34.152648: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-05 04:20:34.154293: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9815 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-03-05 04:20:34.461964: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-05 04:20:34.462086: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-05 04:20:34.462108: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-05 04:20:34.462128: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-05 04:20:34.462147: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-05 04:20:34.462164: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-05 04:20:34.462183: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-05 04:20:34.462201: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-05 04:20:34.463745: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-05 04:20:34.465285: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-05 04:20:34.465325: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-05 04:20:34.465343: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-05 04:20:34.465361: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-05 04:20:34.465378: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-05 04:20:34.465395: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-05 04:20:34.465412: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-05 04:20:34.465429: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-05 04:20:34.466976: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-05 04:20:34.467024: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-05 04:20:34.467036: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-05 04:20:34.467045: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-05 04:20:34.468424: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9815 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-03-05 04:20:42.402456: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-05 04:20:42.571350: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 1 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 37.00000 max: 217.00000 molded_images shape: (1, 640, 640, 3) min: -82.56719 max: 101.79531 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 7.295608520507812e-05 nb_pixel_total : 73 time to create 1 rle with old method : 0.00022339820861816406 length of segment : 13 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.65313 max: 145.38125 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 7.033348083496094e-05 nb_pixel_total : 148 time to create 1 rle with old method : 0.00041556358337402344 length of segment : 26 time for calcul the mask position with numpy : 5.817413330078125e-05 nb_pixel_total : 245 time to create 1 rle with old method : 0.0005216598510742188 length of segment : 20 time for calcul the mask position with numpy : 5.030632019042969e-05 nb_pixel_total : 193 time to create 1 rle with old method : 0.00043201446533203125 length of segment : 22 time for calcul the mask position with numpy : 5.3882598876953125e-05 nb_pixel_total : 252 time to create 1 rle with old method : 0.000560760498046875 length of segment : 22 time for calcul the mask position with numpy : 4.863739013671875e-05 nb_pixel_total : 229 time to create 1 rle with old method : 0.0005524158477783203 length of segment : 21 time for calcul the mask position with numpy : 3.62396240234375e-05 nb_pixel_total : 85 time to create 1 rle with old method : 0.00020170211791992188 length of segment : 10 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -111.88359 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 19 time for calcul the mask position with numpy : 8.749961853027344e-05 nb_pixel_total : 479 time to create 1 rle with old method : 0.0011000633239746094 length of segment : 27 time for calcul the mask position with numpy : 7.510185241699219e-05 nb_pixel_total : 943 time to create 1 rle with old method : 0.0020570755004882812 length of segment : 31 time for calcul the mask position with numpy : 4.458427429199219e-05 nb_pixel_total : 289 time to create 1 rle with old method : 0.000476837158203125 length of segment : 18 time for calcul the mask position with numpy : 4.267692565917969e-05 nb_pixel_total : 538 time to create 1 rle with old method : 0.0008366107940673828 length of segment : 20 time for calcul the mask position with numpy : 4.9114227294921875e-05 nb_pixel_total : 534 time to create 1 rle with old method : 0.0008745193481445312 length of segment : 48 time for calcul the mask position with numpy : 5.8650970458984375e-05 nb_pixel_total : 219 time to create 1 rle with old method : 0.0004782676696777344 length of segment : 13 time for calcul the mask position with numpy : 4.410743713378906e-05 nb_pixel_total : 296 time to create 1 rle with old method : 0.0005795955657958984 length of segment : 22 time for calcul the mask position with numpy : 6.0558319091796875e-05 nb_pixel_total : 961 time to create 1 rle with old method : 0.0017333030700683594 length of segment : 43 time for calcul the mask position with numpy : 6.914138793945312e-05 nb_pixel_total : 2055 time to create 1 rle with old method : 0.0028955936431884766 length of segment : 50 time for calcul the mask position with numpy : 4.029273986816406e-05 nb_pixel_total : 348 time to create 1 rle with old method : 0.0005173683166503906 length of segment : 26 time for calcul the mask position with numpy : 4.00543212890625e-05 nb_pixel_total : 718 time to create 1 rle with old method : 0.0010676383972167969 length of segment : 40 time for calcul the mask position with numpy : 4.291534423828125e-05 nb_pixel_total : 662 time to create 1 rle with old method : 0.0010464191436767578 length of segment : 33 time for calcul the mask position with numpy : 5.1021575927734375e-05 nb_pixel_total : 462 time to create 1 rle with old method : 0.0008966922760009766 length of segment : 24 time for calcul the mask position with numpy : 3.4332275390625e-05 nb_pixel_total : 173 time to create 1 rle with old method : 0.0002944469451904297 length of segment : 14 time for calcul the mask position with numpy : 3.266334533691406e-05 nb_pixel_total : 154 time to create 1 rle with old method : 0.0002760887145996094 length of segment : 11 time for calcul the mask position with numpy : 3.7670135498046875e-05 nb_pixel_total : 479 time to create 1 rle with old method : 0.0008027553558349609 length of segment : 26 time for calcul the mask position with numpy : 2.9325485229492188e-05 nb_pixel_total : 34 time to create 1 rle with old method : 7.796287536621094e-05 length of segment : 8 time for calcul the mask position with numpy : 3.5762786865234375e-05 nb_pixel_total : 204 time to create 1 rle with old method : 0.0004363059997558594 length of segment : 16 time for calcul the mask position with numpy : 4.839897155761719e-05 nb_pixel_total : 690 time to create 1 rle with old method : 0.0010797977447509766 length of segment : 32 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -113.60625 max: 151.02969 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 10 time for calcul the mask position with numpy : 4.410743713378906e-05 nb_pixel_total : 449 time to create 1 rle with old method : 0.0008158683776855469 length of segment : 18 time for calcul the mask position with numpy : 4.029273986816406e-05 nb_pixel_total : 490 time to create 1 rle with old method : 0.0007643699645996094 length of segment : 31 time for calcul the mask position with numpy : 7.033348083496094e-05 nb_pixel_total : 469 time to create 1 rle with old method : 0.0011746883392333984 length of segment : 28 time for calcul the mask position with numpy : 4.029273986816406e-05 nb_pixel_total : 314 time to create 1 rle with old method : 0.0005767345428466797 length of segment : 22 time for calcul the mask position with numpy : 8.392333984375e-05 nb_pixel_total : 1786 time to create 1 rle with old method : 0.0027616024017333984 length of segment : 91 time for calcul the mask position with numpy : 6.461143493652344e-05 nb_pixel_total : 529 time to create 1 rle with old method : 0.0008144378662109375 length of segment : 30 time for calcul the mask position with numpy : 6.747245788574219e-05 nb_pixel_total : 1413 time to create 1 rle with old method : 0.002282857894897461 length of segment : 47 time for calcul the mask position with numpy : 9.059906005859375e-05 nb_pixel_total : 1911 time to create 1 rle with old method : 0.0026776790618896484 length of segment : 79 time for calcul the mask position with numpy : 3.600120544433594e-05 nb_pixel_total : 198 time to create 1 rle with old method : 0.00034308433532714844 length of segment : 17 time for calcul the mask position with numpy : 7.200241088867188e-05 nb_pixel_total : 1997 time to create 1 rle with old method : 0.0029060840606689453 length of segment : 79 Processing 1 images image shape: (400, 400, 3) min: 2.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -94.07891 max: 146.15469 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 286 time to create 1 rle with old method : 0.00045752525329589844 length of segment : 26 time for calcul the mask position with numpy : 5.817413330078125e-05 nb_pixel_total : 510 time to create 1 rle with old method : 0.0010013580322265625 length of segment : 43 time for calcul the mask position with numpy : 5.984306335449219e-05 nb_pixel_total : 575 time to create 1 rle with old method : 0.0011487007141113281 length of segment : 25 Processing 1 images image shape: (400, 400, 3) min: 22.00000 max: 252.00000 molded_images shape: (1, 640, 640, 3) min: -91.08281 max: 137.04531 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 4.553794860839844e-05 nb_pixel_total : 476 time to create 1 rle with old method : 0.0007605552673339844 length of segment : 24 time for calcul the mask position with numpy : 4.792213439941406e-05 nb_pixel_total : 462 time to create 1 rle with old method : 0.0007767677307128906 length of segment : 23 Processing 1 images image shape: (400, 160, 3) min: 36.00000 max: 139.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 23.15859 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 4.553794860839844e-05 nb_pixel_total : 190 time to create 1 rle with old method : 0.0003314018249511719 length of segment : 17 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.73906 max: 150.80703 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 5.53131103515625e-05 nb_pixel_total : 587 time to create 1 rle with old method : 0.001051187515258789 length of segment : 60 time for calcul the mask position with numpy : 5.412101745605469e-05 nb_pixel_total : 555 time to create 1 rle with old method : 0.0008959770202636719 length of segment : 47 time for calcul the mask position with numpy : 6.0558319091796875e-05 nb_pixel_total : 1767 time to create 1 rle with old method : 0.002629518508911133 length of segment : 36 time for calcul the mask position with numpy : 0.00018739700317382812 nb_pixel_total : 2461 time to create 1 rle with old method : 0.0036897659301757812 length of segment : 118 time for calcul the mask position with numpy : 6.008148193359375e-05 nb_pixel_total : 1205 time to create 1 rle with old method : 0.002093791961669922 length of segment : 15 time for calcul the mask position with numpy : 8.416175842285156e-05 nb_pixel_total : 1408 time to create 1 rle with old method : 0.0021686553955078125 length of segment : 89 time for calcul the mask position with numpy : 8.153915405273438e-05 nb_pixel_total : 1018 time to create 1 rle with old method : 0.0017175674438476562 length of segment : 78 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -119.25078 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 15 time for calcul the mask position with numpy : 6.67572021484375e-05 nb_pixel_total : 252 time to create 1 rle with old method : 0.0004107952117919922 length of segment : 15 time for calcul the mask position with numpy : 8.296966552734375e-05 nb_pixel_total : 2238 time to create 1 rle with old method : 0.0034132003784179688 length of segment : 59 time for calcul the mask position with numpy : 0.00028824806213378906 nb_pixel_total : 5897 time to create 1 rle with old method : 0.009894847869873047 length of segment : 201 time for calcul the mask position with numpy : 0.00010752677917480469 nb_pixel_total : 295 time to create 1 rle with old method : 0.0007507801055908203 length of segment : 21 time for calcul the mask position with numpy : 0.0001442432403564453 nb_pixel_total : 1721 time to create 1 rle with old method : 0.004071712493896484 length of segment : 42 time for calcul the mask position with numpy : 9.179115295410156e-05 nb_pixel_total : 2221 time to create 1 rle with old method : 0.003205537796020508 length of segment : 72 time for calcul the mask position with numpy : 0.00011801719665527344 nb_pixel_total : 4374 time to create 1 rle with old method : 0.006171703338623047 length of segment : 89 time for calcul the mask position with numpy : 0.00032210350036621094 nb_pixel_total : 15082 time to create 1 rle with old method : 0.02228403091430664 length of segment : 163 time for calcul the mask position with numpy : 0.00016927719116210938 nb_pixel_total : 3412 time to create 1 rle with old method : 0.004647254943847656 length of segment : 147 time for calcul the mask position with numpy : 0.00028896331787109375 nb_pixel_total : 1146 time to create 1 rle with old method : 0.0015323162078857422 length of segment : 112 time for calcul the mask position with numpy : 7.104873657226562e-05 nb_pixel_total : 1024 time to create 1 rle with old method : 0.0020465850830078125 length of segment : 58 time for calcul the mask position with numpy : 6.437301635742188e-05 nb_pixel_total : 391 time to create 1 rle with old method : 0.0006697177886962891 length of segment : 43 time for calcul the mask position with numpy : 5.221366882324219e-05 nb_pixel_total : 246 time to create 1 rle with old method : 0.00038886070251464844 length of segment : 53 time for calcul the mask position with numpy : 7.43865966796875e-05 nb_pixel_total : 1101 time to create 1 rle with old method : 0.0017387866973876953 length of segment : 58 time for calcul the mask position with numpy : 0.0002281665802001953 nb_pixel_total : 12131 time to create 1 rle with old method : 0.017057418823242188 length of segment : 149 Processing 1 images image shape: (400, 400, 3) min: 4.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -110.19219 max: 150.88125 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 10 time for calcul the mask position with numpy : 6.985664367675781e-05 nb_pixel_total : 849 time to create 1 rle with old method : 0.0012688636779785156 length of segment : 36 time for calcul the mask position with numpy : 4.482269287109375e-05 nb_pixel_total : 230 time to create 1 rle with old method : 0.00045871734619140625 length of segment : 14 time for calcul the mask position with numpy : 4.553794860839844e-05 nb_pixel_total : 426 time to create 1 rle with old method : 0.0006620883941650391 length of segment : 18 time for calcul the mask position with numpy : 7.724761962890625e-05 nb_pixel_total : 2187 time to create 1 rle with old method : 0.0030744075775146484 length of segment : 55 time for calcul the mask position with numpy : 5.9604644775390625e-05 nb_pixel_total : 843 time to create 1 rle with old method : 0.0012362003326416016 length of segment : 50 time for calcul the mask position with numpy : 6.961822509765625e-05 nb_pixel_total : 1980 time to create 1 rle with old method : 0.002866983413696289 length of segment : 51 time for calcul the mask position with numpy : 4.8160552978515625e-05 nb_pixel_total : 317 time to create 1 rle with old method : 0.0005183219909667969 length of segment : 30 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 258 time to create 1 rle with old method : 0.00043320655822753906 length of segment : 30 time for calcul the mask position with numpy : 7.2479248046875e-05 nb_pixel_total : 2128 time to create 1 rle with old method : 0.0030069351196289062 length of segment : 54 time for calcul the mask position with numpy : 6.771087646484375e-05 nb_pixel_total : 1357 time to create 1 rle with old method : 0.002021312713623047 length of segment : 40 Processing 1 images image shape: (400, 400, 3) min: 11.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -95.30156 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 5.078315734863281e-05 nb_pixel_total : 564 time to create 1 rle with old method : 0.0008606910705566406 length of segment : 19 time for calcul the mask position with numpy : 4.38690185546875e-05 nb_pixel_total : 399 time to create 1 rle with old method : 0.000743865966796875 length of segment : 13 time for calcul the mask position with numpy : 4.38690185546875e-05 nb_pixel_total : 185 time to create 1 rle with old method : 0.00032210350036621094 length of segment : 20 time for calcul the mask position with numpy : 5.7697296142578125e-05 nb_pixel_total : 1038 time to create 1 rle with old method : 0.0017242431640625 length of segment : 33 time for calcul the mask position with numpy : 5.221366882324219e-05 nb_pixel_total : 424 time to create 1 rle with old method : 0.0007593631744384766 length of segment : 24 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 70 time to create 1 rle with old method : 0.000362396240234375 length of segment : 13 time for calcul the mask position with numpy : 9.584426879882812e-05 nb_pixel_total : 282 time to create 1 rle with old method : 0.0005171298980712891 length of segment : 23 Processing 1 images image shape: (400, 400, 3) min: 39.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -78.59063 max: 149.10391 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 6.0558319091796875e-05 nb_pixel_total : 1768 time to create 1 rle with old method : 0.0024933815002441406 length of segment : 53 time for calcul the mask position with numpy : 5.0067901611328125e-05 nb_pixel_total : 404 time to create 1 rle with old method : 0.0007534027099609375 length of segment : 34 time for calcul the mask position with numpy : 4.57763671875e-05 nb_pixel_total : 292 time to create 1 rle with old method : 0.0005733966827392578 length of segment : 13 time for calcul the mask position with numpy : 9.083747863769531e-05 nb_pixel_total : 3366 time to create 1 rle with old method : 0.004730224609375 length of segment : 60 time for calcul the mask position with numpy : 7.62939453125e-05 nb_pixel_total : 1965 time to create 1 rle with old method : 0.0027887821197509766 length of segment : 56 time for calcul the mask position with numpy : 7.581710815429688e-05 nb_pixel_total : 1967 time to create 1 rle with old method : 0.002871990203857422 length of segment : 48 time for calcul the mask position with numpy : 4.863739013671875e-05 nb_pixel_total : 506 time to create 1 rle with old method : 0.0007603168487548828 length of segment : 32 time for calcul the mask position with numpy : 5.3882598876953125e-05 nb_pixel_total : 555 time to create 1 rle with old method : 0.0011012554168701172 length of segment : 25 time for calcul the mask position with numpy : 4.7206878662109375e-05 nb_pixel_total : 549 time to create 1 rle with old method : 0.0008158683776855469 length of segment : 27 Processing 1 images image shape: (400, 400, 3) min: 1.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -119.24297 max: 151.07266 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 14 time for calcul the mask position with numpy : 7.128715515136719e-05 nb_pixel_total : 1304 time to create 1 rle with old method : 0.0020973682403564453 length of segment : 61 time for calcul the mask position with numpy : 5.888938903808594e-05 nb_pixel_total : 617 time to create 1 rle with old method : 0.0009853839874267578 length of segment : 32 time for calcul the mask position with numpy : 4.6253204345703125e-05 nb_pixel_total : 350 time to create 1 rle with old method : 0.0007035732269287109 length of segment : 21 time for calcul the mask position with numpy : 4.9114227294921875e-05 nb_pixel_total : 359 time to create 1 rle with old method : 0.0005588531494140625 length of segment : 22 time for calcul the mask position with numpy : 0.00011563301086425781 nb_pixel_total : 1120 time to create 1 rle with old method : 0.0018420219421386719 length of segment : 101 time for calcul the mask position with numpy : 4.863739013671875e-05 nb_pixel_total : 178 time to create 1 rle with old method : 0.0003459453582763672 length of segment : 15 time for calcul the mask position with numpy : 5.936622619628906e-05 nb_pixel_total : 1263 time to create 1 rle with old method : 0.001895904541015625 length of segment : 51 time for calcul the mask position with numpy : 5.793571472167969e-05 nb_pixel_total : 511 time to create 1 rle with old method : 0.0008685588836669922 length of segment : 32 time for calcul the mask position with numpy : 4.7206878662109375e-05 nb_pixel_total : 422 time to create 1 rle with old method : 0.0007321834564208984 length of segment : 27 time for calcul the mask position with numpy : 5.054473876953125e-05 nb_pixel_total : 614 time to create 1 rle with old method : 0.0009493827819824219 length of segment : 29 time for calcul the mask position with numpy : 6.67572021484375e-05 nb_pixel_total : 126 time to create 1 rle with old method : 0.00044798851013183594 length of segment : 14 time for calcul the mask position with numpy : 5.0067901611328125e-05 nb_pixel_total : 321 time to create 1 rle with old method : 0.0005388259887695312 length of segment : 25 time for calcul the mask position with numpy : 6.079673767089844e-05 nb_pixel_total : 887 time to create 1 rle with old method : 0.0013804435729980469 length of segment : 44 time for calcul the mask position with numpy : 6.031990051269531e-05 nb_pixel_total : 930 time to create 1 rle with old method : 0.0014333724975585938 length of segment : 38 Processing 1 images image shape: (400, 160, 3) min: 14.00000 max: 221.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 87.97578 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 5.054473876953125e-05 nb_pixel_total : 492 time to create 1 rle with old method : 0.0007622241973876953 length of segment : 28 time for calcul the mask position with numpy : 4.172325134277344e-05 nb_pixel_total : 196 time to create 1 rle with old method : 0.00034618377685546875 length of segment : 13 time for calcul the mask position with numpy : 0.0001938343048095703 nb_pixel_total : 11457 time to create 1 rle with old method : 0.015994548797607422 length of segment : 272 Processing 1 images image shape: (400, 400, 3) min: 14.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -104.68438 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 5.507469177246094e-05 nb_pixel_total : 319 time to create 1 rle with old method : 0.0004858970642089844 length of segment : 27 time for calcul the mask position with numpy : 5.984306335449219e-05 nb_pixel_total : 1037 time to create 1 rle with old method : 0.0014815330505371094 length of segment : 40 time for calcul the mask position with numpy : 5.340576171875e-05 nb_pixel_total : 672 time to create 1 rle with old method : 0.0009813308715820312 length of segment : 54 time for calcul the mask position with numpy : 8.559226989746094e-05 nb_pixel_total : 1921 time to create 1 rle with old method : 0.0029528141021728516 length of segment : 84 time for calcul the mask position with numpy : 5.8650970458984375e-05 nb_pixel_total : 389 time to create 1 rle with old method : 0.0007174015045166016 length of segment : 23 time for calcul the mask position with numpy : 0.00011777877807617188 nb_pixel_total : 4156 time to create 1 rle with old method : 0.006116151809692383 length of segment : 98 time for calcul the mask position with numpy : 5.8650970458984375e-05 nb_pixel_total : 639 time to create 1 rle with old method : 0.0010232925415039062 length of segment : 34 time for calcul the mask position with numpy : 5.817413330078125e-05 nb_pixel_total : 777 time to create 1 rle with old method : 0.0012340545654296875 length of segment : 29 time for calcul the mask position with numpy : 8.440017700195312e-05 nb_pixel_total : 1366 time to create 1 rle with old method : 0.0020494461059570312 length of segment : 91 Processing 1 images image shape: (400, 400, 3) min: 2.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -114.33281 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 11 time for calcul the mask position with numpy : 5.0067901611328125e-05 nb_pixel_total : 383 time to create 1 rle with old method : 0.0006215572357177734 length of segment : 45 time for calcul the mask position with numpy : 8.916854858398438e-05 nb_pixel_total : 3417 time to create 1 rle with old method : 0.005078554153442383 length of segment : 55 time for calcul the mask position with numpy : 6.389617919921875e-05 nb_pixel_total : 2131 time to create 1 rle with old method : 0.0033979415893554688 length of segment : 49 time for calcul the mask position with numpy : 6.151199340820312e-05 nb_pixel_total : 421 time to create 1 rle with old method : 0.0007107257843017578 length of segment : 39 time for calcul the mask position with numpy : 5.626678466796875e-05 nb_pixel_total : 1179 time to create 1 rle with old method : 0.002023458480834961 length of segment : 34 time for calcul the mask position with numpy : 4.291534423828125e-05 nb_pixel_total : 528 time to create 1 rle with old method : 0.0008599758148193359 length of segment : 53 time for calcul the mask position with numpy : 0.00047135353088378906 nb_pixel_total : 18589 time to create 1 rle with old method : 0.02697610855102539 length of segment : 519 time for calcul the mask position with numpy : 0.00010132789611816406 nb_pixel_total : 2427 time to create 1 rle with old method : 0.006335258483886719 length of segment : 62 time for calcul the mask position with numpy : 0.00015282630920410156 nb_pixel_total : 4708 time to create 1 rle with old method : 0.006937265396118164 length of segment : 146 time for calcul the mask position with numpy : 8.082389831542969e-05 nb_pixel_total : 2093 time to create 1 rle with old method : 0.0032422542572021484 length of segment : 54 time for calcul the mask position with numpy : 0.0012595653533935547 nb_pixel_total : 71664 time to create 1 rle with old method : 0.10731720924377441 length of segment : 526 Processing 1 images image shape: (400, 400, 3) min: 20.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -93.71563 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 4.410743713378906e-05 nb_pixel_total : 263 time to create 1 rle with old method : 0.0004820823669433594 length of segment : 17 time for calcul the mask position with numpy : 0.0002155303955078125 nb_pixel_total : 10731 time to create 1 rle with old method : 0.014917373657226562 length of segment : 204 time for calcul the mask position with numpy : 0.00010657310485839844 nb_pixel_total : 2971 time to create 1 rle with old method : 0.0040891170501708984 length of segment : 75 time for calcul the mask position with numpy : 4.1961669921875e-05 nb_pixel_total : 487 time to create 1 rle with old method : 0.0007665157318115234 length of segment : 23 time for calcul the mask position with numpy : 6.604194641113281e-05 nb_pixel_total : 942 time to create 1 rle with old method : 0.0013611316680908203 length of segment : 70 time for calcul the mask position with numpy : 6.031990051269531e-05 nb_pixel_total : 1328 time to create 1 rle with old method : 0.0019674301147460938 length of segment : 64 Processing 1 images image shape: (400, 400, 3) min: 52.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -65.73516 max: 143.33047 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 384 time to create 1 rle with old method : 0.0006372928619384766 length of segment : 24 time for calcul the mask position with numpy : 0.0006854534149169922 nb_pixel_total : 57170 time to create 1 rle with old method : 0.08237099647521973 length of segment : 268 time for calcul the mask position with numpy : 0.0001304149627685547 nb_pixel_total : 2691 time to create 1 rle with old method : 0.0039539337158203125 length of segment : 90 time for calcul the mask position with numpy : 0.00013589859008789062 nb_pixel_total : 1027 time to create 1 rle with old method : 0.0016026496887207031 length of segment : 77 time for calcul the mask position with numpy : 3.695487976074219e-05 nb_pixel_total : 70 time to create 1 rle with old method : 0.00015997886657714844 length of segment : 6 Processing 1 images image shape: (400, 400, 3) min: 53.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -70.70000 max: 146.17422 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0006525516510009766 nb_pixel_total : 64629 time to create 1 rle with old method : 0.08578205108642578 length of segment : 313 Processing 1 images image shape: (400, 400, 3) min: 12.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -95.80156 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 9.942054748535156e-05 nb_pixel_total : 3990 time to create 1 rle with old method : 0.005898237228393555 length of segment : 63 time for calcul the mask position with numpy : 6.175041198730469e-05 nb_pixel_total : 881 time to create 1 rle with old method : 0.00135040283203125 length of segment : 65 time for calcul the mask position with numpy : 6.556510925292969e-05 nb_pixel_total : 721 time to create 1 rle with old method : 0.001196146011352539 length of segment : 33 time for calcul the mask position with numpy : 5.054473876953125e-05 nb_pixel_total : 372 time to create 1 rle with old method : 0.0008051395416259766 length of segment : 11 time for calcul the mask position with numpy : 9.5367431640625e-05 nb_pixel_total : 2105 time to create 1 rle with old method : 0.003144979476928711 length of segment : 99 time for calcul the mask position with numpy : 4.863739013671875e-05 nb_pixel_total : 425 time to create 1 rle with old method : 0.0006976127624511719 length of segment : 31 time for calcul the mask position with numpy : 4.57763671875e-05 nb_pixel_total : 703 time to create 1 rle with old method : 0.0011279582977294922 length of segment : 28 Processing 1 images image shape: (400, 160, 3) min: 12.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 136.31719 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 5.8650970458984375e-05 nb_pixel_total : 1544 time to create 1 rle with old method : 0.0020062923431396484 length of segment : 108 time for calcul the mask position with numpy : 3.981590270996094e-05 nb_pixel_total : 109 time to create 1 rle with old method : 0.0002942085266113281 length of segment : 21 time for calcul the mask position with numpy : 3.838539123535156e-05 nb_pixel_total : 695 time to create 1 rle with old method : 0.0010404586791992188 length of segment : 26 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 797 time to create 1 rle with old method : 0.001207113265991211 length of segment : 40 time for calcul the mask position with numpy : 0.00017786026000976562 nb_pixel_total : 3635 time to create 1 rle with old method : 0.0050373077392578125 length of segment : 264 time for calcul the mask position with numpy : 5.555152893066406e-05 nb_pixel_total : 823 time to create 1 rle with old method : 0.001230478286743164 length of segment : 39 Processing 1 images image shape: (240, 400, 3) min: 61.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 5.888938903808594e-05 nb_pixel_total : 254 time to create 1 rle with old method : 0.0005216598510742188 length of segment : 25 time for calcul the mask position with numpy : 6.818771362304688e-05 nb_pixel_total : 942 time to create 1 rle with old method : 0.0017933845520019531 length of segment : 58 Processing 1 images image shape: (240, 400, 3) min: 44.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 5.173683166503906e-05 nb_pixel_total : 317 time to create 1 rle with old method : 0.0004799365997314453 length of segment : 56 time for calcul the mask position with numpy : 6.29425048828125e-05 nb_pixel_total : 1112 time to create 1 rle with old method : 0.0016148090362548828 length of segment : 64 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 502 time to create 1 rle with old method : 0.0008032321929931641 length of segment : 30 time for calcul the mask position with numpy : 3.9577484130859375e-05 nb_pixel_total : 172 time to create 1 rle with old method : 0.00032067298889160156 length of segment : 28 time for calcul the mask position with numpy : 0.00011897087097167969 nb_pixel_total : 5085 time to create 1 rle with old method : 0.007246255874633789 length of segment : 93 Processing 1 images image shape: (240, 400, 3) min: 29.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 149.38516 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 5.14984130859375e-05 nb_pixel_total : 3 time to create 1 rle with old method : 4.458427429199219e-05 length of segment : 2 Processing 1 images image shape: (240, 400, 3) min: 54.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 147.63906 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 4.792213439941406e-05 nb_pixel_total : 926 time to create 1 rle with old method : 0.0012578964233398438 length of segment : 46 Processing 1 images image shape: (240, 400, 3) min: 44.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 135.92812 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 5.2928924560546875e-05 nb_pixel_total : 793 time to create 1 rle with old method : 0.001116037368774414 length of segment : 35 Processing 1 images image shape: (240, 400, 3) min: 28.00000 max: 207.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 102.26797 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 5.054473876953125e-05 nb_pixel_total : 800 time to create 1 rle with old method : 0.0011706352233886719 length of segment : 72 time for calcul the mask position with numpy : 0.00024008750915527344 nb_pixel_total : 16016 time to create 1 rle with old method : 0.020761966705322266 length of segment : 117 Processing 1 images image shape: (240, 160, 3) min: 65.00000 max: 182.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 76.19427 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Detection mask done ! Trying to reset tf kernel 3215791 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 4505 tf kernel not reseted sub process len(results) : 164 len(list_Values) 164 None max_time_sub_proc : 3600 parent process len(results) : 0 len(list_Values) 164 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 9794 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Catched exception ! Connect or reconnect ! Number saved : None batch 1 Loaded 199 chid ids of type : 4228 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 9404 save missing photos in datou_result : time spend for datou_step_exec : 23.864750862121582 time spend to save output : 7.321581840515137 total time spend for step 1 : 31.18633270263672 step2:brightness Wed Mar 5 04:21:01 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness toutes les photos sont déjà traitées, on saute les calculs Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 1 time used for this insertion : 0.008986711502075195 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 1 time used for this insertion : 0.007830619812011719 save missing photos in datou_result : time spend for datou_step_exec : 0.026564359664916992 time spend to save output : 0.023916244506835938 total time spend for step 2 : 0.05048060417175293 step3:blur_detection Wed Mar 5 04:21:01 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection toutes les photos sont déjà traitées, on saute les calculs Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 1 time used for this insertion : 0.00793910026550293 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 1 time used for this insertion : 0.008967161178588867 save missing photos in datou_result : time spend for datou_step_exec : 0.02257704734802246 time spend to save output : 0.021436691284179688 total time spend for step 3 : 0.04401373863220215 step4:rle_unique_nms_with_priority Wed Mar 5 04:21:01 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 199 chid ids of type : 4228 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 198 nb_hashtags : 5 time to prepare the origin masks : 9.156959533691406 time for calcul the mask position with numpy : 0.35213661193847656 nb_pixel_total : 3270575 time to create 1 rle with new method : 0.33669066429138184 time for calcul the mask position with numpy : 0.01487112045288086 nb_pixel_total : 34 time to create 1 rle with old method : 7.009506225585938e-05 time for calcul the mask position with numpy : 0.014645099639892578 nb_pixel_total : 296 time to create 1 rle with old method : 0.00037097930908203125 time for calcul the mask position with numpy : 0.014619588851928711 nb_pixel_total : 73 time to create 1 rle with old method : 0.00011014938354492188 time for calcul the mask position with numpy : 0.014816522598266602 nb_pixel_total : 219 time to create 1 rle with old method : 0.00029397010803222656 time for calcul the mask position with numpy : 0.014775753021240234 nb_pixel_total : 289 time to create 1 rle with old method : 0.0003886222839355469 time for calcul the mask position with numpy : 0.014951944351196289 nb_pixel_total : 961 time to create 1 rle with old method : 0.0012650489807128906 time for calcul the mask position with numpy : 0.014905691146850586 nb_pixel_total : 469 time to create 1 rle with old method : 0.0006086826324462891 time for calcul the mask position with numpy : 0.014883756637573242 nb_pixel_total : 252 time to create 1 rle with old method : 0.0003330707550048828 time for calcul the mask position with numpy : 0.014876127243041992 nb_pixel_total : 11 time to create 1 rle with old method : 4.4345855712890625e-05 time for calcul the mask position with numpy : 0.014900922775268555 nb_pixel_total : 148 time to create 1 rle with old method : 0.00020170211791992188 time for calcul the mask position with numpy : 0.01485300064086914 nb_pixel_total : 85 time to create 1 rle with old method : 0.0001780986785888672 time for calcul the mask position with numpy : 0.015109539031982422 nb_pixel_total : 534 time to create 1 rle with old method : 0.0006835460662841797 time for calcul the mask position with numpy : 0.01472020149230957 nb_pixel_total : 198 time to create 1 rle with old method : 0.0002694129943847656 time for calcul the mask position with numpy : 0.014741659164428711 nb_pixel_total : 204 time to create 1 rle with old method : 0.0002808570861816406 time for calcul the mask position with numpy : 0.014827489852905273 nb_pixel_total : 943 time to create 1 rle with old method : 0.001260995864868164 time for calcul the mask position with numpy : 0.01443028450012207 nb_pixel_total : 190 time to create 1 rle with old method : 0.0002422332763671875 time for calcul the mask position with numpy : 0.014573097229003906 nb_pixel_total : 1786 time to create 1 rle with old method : 0.0022177696228027344 time for calcul the mask position with numpy : 0.014442205429077148 nb_pixel_total : 538 time to create 1 rle with old method : 0.0006949901580810547 time for calcul the mask position with numpy : 0.014493703842163086 nb_pixel_total : 449 time to create 1 rle with old method : 0.0005676746368408203 time for calcul the mask position with numpy : 0.014254570007324219 nb_pixel_total : 173 time to create 1 rle with old method : 0.00022745132446289062 time for calcul the mask position with numpy : 0.01634955406188965 nb_pixel_total : 1413 time to create 1 rle with old method : 0.0017342567443847656 time for calcul the mask position with numpy : 0.01429438591003418 nb_pixel_total : 510 time to create 1 rle with old method : 0.0007340908050537109 time for calcul the mask position with numpy : 0.014380216598510742 nb_pixel_total : 314 time to create 1 rle with old method : 0.00041747093200683594 time for calcul the mask position with numpy : 0.014519214630126953 nb_pixel_total : 42 time to create 1 rle with old method : 8.559226989746094e-05 time for calcul the mask position with numpy : 0.014477014541625977 nb_pixel_total : 490 time to create 1 rle with old method : 0.0005815029144287109 time for calcul the mask position with numpy : 0.014419794082641602 nb_pixel_total : 479 time to create 1 rle with old method : 0.0006155967712402344 time for calcul the mask position with numpy : 0.01454615592956543 nb_pixel_total : 348 time to create 1 rle with old method : 0.00045752525329589844 time for calcul the mask position with numpy : 0.014357328414916992 nb_pixel_total : 154 time to create 1 rle with old method : 0.00021791458129882812 time for calcul the mask position with numpy : 0.014501571655273438 nb_pixel_total : 193 time to create 1 rle with old method : 0.0002601146697998047 time for calcul the mask position with numpy : 0.01462244987487793 nb_pixel_total : 462 time to create 1 rle with old method : 0.0006406307220458984 time for calcul the mask position with numpy : 0.014383077621459961 nb_pixel_total : 575 time to create 1 rle with old method : 0.0007269382476806641 time for calcul the mask position with numpy : 0.014294147491455078 nb_pixel_total : 2055 time to create 1 rle with old method : 0.0026035308837890625 time for calcul the mask position with numpy : 0.014437675476074219 nb_pixel_total : 69 time to create 1 rle with old method : 0.0001728534698486328 time for calcul the mask position with numpy : 0.014796733856201172 nb_pixel_total : 10 time to create 1 rle with old method : 4.76837158203125e-05 time for calcul the mask position with numpy : 0.014951467514038086 nb_pixel_total : 215 time to create 1 rle with old method : 0.0003361701965332031 time for calcul the mask position with numpy : 0.01657891273498535 nb_pixel_total : 1911 time to create 1 rle with old method : 0.0023915767669677734 time for calcul the mask position with numpy : 0.014701604843139648 nb_pixel_total : 476 time to create 1 rle with old method : 0.0005970001220703125 time for calcul the mask position with numpy : 0.014890909194946289 nb_pixel_total : 11 time to create 1 rle with old method : 7.05718994140625e-05 time for calcul the mask position with numpy : 0.015276670455932617 nb_pixel_total : 662 time to create 1 rle with old method : 0.0008008480072021484 time for calcul the mask position with numpy : 0.01490926742553711 nb_pixel_total : 41 time to create 1 rle with old method : 0.00011873245239257812 time for calcul the mask position with numpy : 0.016761064529418945 nb_pixel_total : 229 time to create 1 rle with old method : 0.0004591941833496094 time for calcul the mask position with numpy : 0.017363309860229492 nb_pixel_total : 286 time to create 1 rle with old method : 0.0005483627319335938 time for calcul the mask position with numpy : 0.017290353775024414 nb_pixel_total : 245 time to create 1 rle with old method : 0.00045800209045410156 time for calcul the mask position with numpy : 0.0174868106842041 nb_pixel_total : 399 time to create 1 rle with old method : 0.0008745193481445312 time for calcul the mask position with numpy : 0.016827106475830078 nb_pixel_total : 350 time to create 1 rle with old method : 0.0005049705505371094 time for calcul the mask position with numpy : 0.015464305877685547 nb_pixel_total : 290 time to create 1 rle with old method : 0.0004062652587890625 time for calcul the mask position with numpy : 0.017234325408935547 nb_pixel_total : 617 time to create 1 rle with old method : 0.0008332729339599609 time for calcul the mask position with numpy : 0.01523447036743164 nb_pixel_total : 7 time to create 1 rle with old method : 3.814697265625e-05 time for calcul the mask position with numpy : 0.015191793441772461 nb_pixel_total : 1357 time to create 1 rle with old method : 0.0017516613006591797 time for calcul the mask position with numpy : 0.015943050384521484 nb_pixel_total : 282 time to create 1 rle with old method : 0.0004284381866455078 time for calcul the mask position with numpy : 0.016022920608520508 nb_pixel_total : 1263 time to create 1 rle with old method : 0.001659393310546875 time for calcul the mask position with numpy : 0.015874862670898438 nb_pixel_total : 4 time to create 1 rle with old method : 6.031990051269531e-05 time for calcul the mask position with numpy : 0.015959501266479492 nb_pixel_total : 8 time to create 1 rle with old method : 8.153915405273438e-05 time for calcul the mask position with numpy : 0.015671968460083008 nb_pixel_total : 549 time to create 1 rle with old method : 0.0007207393646240234 time for calcul the mask position with numpy : 0.014496326446533203 nb_pixel_total : 246 time to create 1 rle with old method : 0.00042557716369628906 time for calcul the mask position with numpy : 0.014587163925170898 nb_pixel_total : 424 time to create 1 rle with old method : 0.0005476474761962891 time for calcul the mask position with numpy : 0.014832735061645508 nb_pixel_total : 15 time to create 1 rle with old method : 3.457069396972656e-05 time for calcul the mask position with numpy : 0.014464616775512695 nb_pixel_total : 47 time to create 1 rle with old method : 0.0001232624053955078 time for calcul the mask position with numpy : 0.014645099639892578 nb_pixel_total : 2213 time to create 1 rle with old method : 0.0027370452880859375 time for calcul the mask position with numpy : 0.014595508575439453 nb_pixel_total : 292 time to create 1 rle with old method : 0.0004734992980957031 time for calcul the mask position with numpy : 0.014507055282592773 nb_pixel_total : 185 time to create 1 rle with old method : 0.00023126602172851562 time for calcul the mask position with numpy : 0.017791748046875 nb_pixel_total : 614 time to create 1 rle with old method : 0.0010297298431396484 time for calcul the mask position with numpy : 0.016138076782226562 nb_pixel_total : 317 time to create 1 rle with old method : 0.0003898143768310547 time for calcul the mask position with numpy : 0.014800548553466797 nb_pixel_total : 422 time to create 1 rle with old method : 0.0005381107330322266 time for calcul the mask position with numpy : 0.01476144790649414 nb_pixel_total : 555 time to create 1 rle with old method : 0.0007052421569824219 time for calcul the mask position with numpy : 0.014671564102172852 nb_pixel_total : 178 time to create 1 rle with old method : 0.0002498626708984375 time for calcul the mask position with numpy : 0.014579057693481445 nb_pixel_total : 849 time to create 1 rle with old method : 0.0010685920715332031 time for calcul the mask position with numpy : 0.014739513397216797 nb_pixel_total : 1967 time to create 1 rle with old method : 0.0024793148040771484 time for calcul the mask position with numpy : 0.014917135238647461 nb_pixel_total : 1467 time to create 1 rle with old method : 0.001844644546508789 time for calcul the mask position with numpy : 0.015119075775146484 nb_pixel_total : 843 time to create 1 rle with old method : 0.0010797977447509766 time for calcul the mask position with numpy : 0.01512908935546875 nb_pixel_total : 321 time to create 1 rle with old method : 0.00042176246643066406 time for calcul the mask position with numpy : 0.015101909637451172 nb_pixel_total : 1120 time to create 1 rle with old method : 0.0014774799346923828 time for calcul the mask position with numpy : 0.015315055847167969 nb_pixel_total : 29 time to create 1 rle with old method : 8.320808410644531e-05 time for calcul the mask position with numpy : 0.015135765075683594 nb_pixel_total : 507 time to create 1 rle with old method : 0.0006306171417236328 time for calcul the mask position with numpy : 0.01800370216369629 nb_pixel_total : 126 time to create 1 rle with old method : 0.00021791458129882812 time for calcul the mask position with numpy : 0.01498866081237793 nb_pixel_total : 426 time to create 1 rle with old method : 0.0005345344543457031 time for calcul the mask position with numpy : 0.014551877975463867 nb_pixel_total : 359 time to create 1 rle with old method : 0.0005078315734863281 time for calcul the mask position with numpy : 0.014305591583251953 nb_pixel_total : 5897 time to create 1 rle with old method : 0.007331371307373047 time for calcul the mask position with numpy : 0.014472007751464844 nb_pixel_total : 11457 time to create 1 rle with old method : 0.014336347579956055 time for calcul the mask position with numpy : 0.01462554931640625 nb_pixel_total : 1790 time to create 1 rle with old method : 0.0025289058685302734 time for calcul the mask position with numpy : 0.014471769332885742 nb_pixel_total : 421 time to create 1 rle with old method : 0.0005474090576171875 time for calcul the mask position with numpy : 0.014660835266113281 nb_pixel_total : 567 time to create 1 rle with old method : 0.0008573532104492188 time for calcul the mask position with numpy : 0.014767885208129883 nb_pixel_total : 2187 time to create 1 rle with old method : 0.002748250961303711 time for calcul the mask position with numpy : 0.014353752136230469 nb_pixel_total : 34 time to create 1 rle with old method : 8.606910705566406e-05 time for calcul the mask position with numpy : 0.014548778533935547 nb_pixel_total : 6 time to create 1 rle with old method : 2.288818359375e-05 time for calcul the mask position with numpy : 0.014309406280517578 nb_pixel_total : 1024 time to create 1 rle with old method : 0.001314401626586914 time for calcul the mask position with numpy : 0.014512777328491211 nb_pixel_total : 96 time to create 1 rle with old method : 0.00016832351684570312 time for calcul the mask position with numpy : 0.014342308044433594 nb_pixel_total : 15 time to create 1 rle with old method : 6.222724914550781e-05 time for calcul the mask position with numpy : 0.014371156692504883 nb_pixel_total : 1965 time to create 1 rle with old method : 0.002419710159301758 time for calcul the mask position with numpy : 0.014791488647460938 nb_pixel_total : 35 time to create 1 rle with old method : 6.29425048828125e-05 time for calcul the mask position with numpy : 0.014590024948120117 nb_pixel_total : 1768 time to create 1 rle with old method : 0.002108335494995117 time for calcul the mask position with numpy : 0.01454472541809082 nb_pixel_total : 1038 time to create 1 rle with old method : 0.0012233257293701172 time for calcul the mask position with numpy : 0.014551401138305664 nb_pixel_total : 7 time to create 1 rle with old method : 4.9114227294921875e-05 time for calcul the mask position with numpy : 0.014671564102172852 nb_pixel_total : 23 time to create 1 rle with old method : 7.343292236328125e-05 time for calcul the mask position with numpy : 0.014394760131835938 nb_pixel_total : 90 time to create 1 rle with old method : 0.00012159347534179688 time for calcul the mask position with numpy : 0.014667272567749023 nb_pixel_total : 1017 time to create 1 rle with old method : 0.0013744831085205078 time for calcul the mask position with numpy : 0.014401912689208984 nb_pixel_total : 389 time to create 1 rle with old method : 0.00063323974609375 time for calcul the mask position with numpy : 0.01447296142578125 nb_pixel_total : 15082 time to create 1 rle with old method : 0.018330812454223633 time for calcul the mask position with numpy : 0.014733314514160156 nb_pixel_total : 258 time to create 1 rle with old method : 0.0003228187561035156 time for calcul the mask position with numpy : 0.01488804817199707 nb_pixel_total : 299 time to create 1 rle with old method : 0.0006186962127685547 time for calcul the mask position with numpy : 0.016989946365356445 nb_pixel_total : 2461 time to create 1 rle with old method : 0.004475831985473633 time for calcul the mask position with numpy : 0.016971349716186523 nb_pixel_total : 506 time to create 1 rle with old method : 0.0009031295776367188 time for calcul the mask position with numpy : 0.017093658447265625 nb_pixel_total : 61 time to create 1 rle with old method : 0.0001983642578125 time for calcul the mask position with numpy : 0.017171859741210938 nb_pixel_total : 3310 time to create 1 rle with old method : 0.005756378173828125 time for calcul the mask position with numpy : 0.016913175582885742 nb_pixel_total : 230 time to create 1 rle with old method : 0.0004458427429199219 time for calcul the mask position with numpy : 0.01667475700378418 nb_pixel_total : 257 time to create 1 rle with old method : 0.0004425048828125 time for calcul the mask position with numpy : 0.016735076904296875 nb_pixel_total : 493 time to create 1 rle with old method : 0.0007059574127197266 time for calcul the mask position with numpy : 0.014879226684570312 nb_pixel_total : 1980 time to create 1 rle with old method : 0.002451181411743164 time for calcul the mask position with numpy : 0.0149383544921875 nb_pixel_total : 555 time to create 1 rle with old method : 0.0007042884826660156 time for calcul the mask position with numpy : 0.015284061431884766 nb_pixel_total : 564 time to create 1 rle with old method : 0.0007066726684570312 time for calcul the mask position with numpy : 0.014908552169799805 nb_pixel_total : 76 time to create 1 rle with old method : 0.00017523765563964844 time for calcul the mask position with numpy : 0.014911174774169922 nb_pixel_total : 887 time to create 1 rle with old method : 0.001129150390625 time for calcul the mask position with numpy : 0.014876365661621094 nb_pixel_total : 2209 time to create 1 rle with old method : 0.00270843505859375 time for calcul the mask position with numpy : 0.01491236686706543 nb_pixel_total : 115 time to create 1 rle with old method : 0.00019359588623046875 time for calcul the mask position with numpy : 0.014958858489990234 nb_pixel_total : 244 time to create 1 rle with old method : 0.0003142356872558594 time for calcul the mask position with numpy : 0.014916658401489258 nb_pixel_total : 582 time to create 1 rle with old method : 0.0007669925689697266 time for calcul the mask position with numpy : 0.01480555534362793 nb_pixel_total : 334 time to create 1 rle with old method : 0.0004398822784423828 time for calcul the mask position with numpy : 0.0149078369140625 nb_pixel_total : 1304 time to create 1 rle with old method : 0.0016684532165527344 time for calcul the mask position with numpy : 0.014903783798217773 nb_pixel_total : 4582 time to create 1 rle with old method : 0.005654811859130859 time for calcul the mask position with numpy : 0.014918804168701172 nb_pixel_total : 48 time to create 1 rle with old method : 0.00010371208190917969 time for calcul the mask position with numpy : 0.014922380447387695 nb_pixel_total : 281 time to create 1 rle with old method : 0.00045609474182128906 time for calcul the mask position with numpy : 0.014964103698730469 nb_pixel_total : 511 time to create 1 rle with old method : 0.0006341934204101562 time for calcul the mask position with numpy : 0.015007257461547852 nb_pixel_total : 1721 time to create 1 rle with old method : 0.002173185348510742 time for calcul the mask position with numpy : 0.015364408493041992 nb_pixel_total : 1767 time to create 1 rle with old method : 0.002651691436767578 time for calcul the mask position with numpy : 0.014739751815795898 nb_pixel_total : 46 time to create 1 rle with old method : 0.00014019012451171875 time for calcul the mask position with numpy : 0.014759063720703125 nb_pixel_total : 404 time to create 1 rle with old method : 0.0005693435668945312 time for calcul the mask position with numpy : 0.014646768569946289 nb_pixel_total : 359 time to create 1 rle with old method : 0.00045299530029296875 time for calcul the mask position with numpy : 0.01472926139831543 nb_pixel_total : 1205 time to create 1 rle with old method : 0.0015244483947753906 time for calcul the mask position with numpy : 0.014460563659667969 nb_pixel_total : 11 time to create 1 rle with old method : 3.4332275390625e-05 time for calcul the mask position with numpy : 0.014625310897827148 nb_pixel_total : 208 time to create 1 rle with old method : 0.0002574920654296875 time for calcul the mask position with numpy : 0.014873981475830078 nb_pixel_total : 70 time to create 1 rle with old method : 0.00014925003051757812 time for calcul the mask position with numpy : 0.01511693000793457 nb_pixel_total : 70 time to create 1 rle with old method : 0.00010728836059570312 time for calcul the mask position with numpy : 0.014978647232055664 nb_pixel_total : 695 time to create 1 rle with old method : 0.0008819103240966797 time for calcul the mask position with numpy : 0.01493072509765625 nb_pixel_total : 881 time to create 1 rle with old method : 0.0011260509490966797 time for calcul the mask position with numpy : 0.015509366989135742 nb_pixel_total : 64629 time to create 1 rle with old method : 0.07737040519714355 time for calcul the mask position with numpy : 0.014961004257202148 nb_pixel_total : 1921 time to create 1 rle with old method : 0.002471923828125 time for calcul the mask position with numpy : 0.014899969100952148 nb_pixel_total : 503 time to create 1 rle with old method : 0.0006606578826904297 time for calcul the mask position with numpy : 0.014740705490112305 nb_pixel_total : 12057 time to create 1 rle with old method : 0.014437198638916016 time for calcul the mask position with numpy : 0.014727354049682617 nb_pixel_total : 1285 time to create 1 rle with old method : 0.001678466796875 time for calcul the mask position with numpy : 0.014911651611328125 nb_pixel_total : 421 time to create 1 rle with old method : 0.0005762577056884766 time for calcul the mask position with numpy : 0.01491999626159668 nb_pixel_total : 109 time to create 1 rle with old method : 0.00015091896057128906 time for calcul the mask position with numpy : 0.016318798065185547 nb_pixel_total : 797 time to create 1 rle with old method : 0.00152587890625 time for calcul the mask position with numpy : 0.01590418815612793 nb_pixel_total : 58 time to create 1 rle with old method : 0.00013303756713867188 time for calcul the mask position with numpy : 0.014826059341430664 nb_pixel_total : 65 time to create 1 rle with old method : 0.00010776519775390625 time for calcul the mask position with numpy : 0.014618873596191406 nb_pixel_total : 2131 time to create 1 rle with old method : 0.0027840137481689453 time for calcul the mask position with numpy : 0.014723062515258789 nb_pixel_total : 388 time to create 1 rle with old method : 0.0008099079132080078 time for calcul the mask position with numpy : 0.015002965927124023 nb_pixel_total : 59214 time to create 1 rle with old method : 0.0752556324005127 time for calcul the mask position with numpy : 0.015449047088623047 nb_pixel_total : 2093 time to create 1 rle with old method : 0.0026657581329345703 time for calcul the mask position with numpy : 0.014976978302001953 nb_pixel_total : 2920 time to create 1 rle with old method : 0.0037927627563476562 time for calcul the mask position with numpy : 0.014131307601928711 nb_pixel_total : 218 time to create 1 rle with old method : 0.00041937828063964844 time for calcul the mask position with numpy : 0.014827251434326172 nb_pixel_total : 18606 time to create 1 rle with old method : 0.022475481033325195 time for calcul the mask position with numpy : 0.014498710632324219 nb_pixel_total : 1720 time to create 1 rle with old method : 0.0021140575408935547 time for calcul the mask position with numpy : 0.014538049697875977 nb_pixel_total : 384 time to create 1 rle with old method : 0.0005762577056884766 time for calcul the mask position with numpy : 0.014749526977539062 nb_pixel_total : 1179 time to create 1 rle with old method : 0.0014715194702148438 time for calcul the mask position with numpy : 0.015658855438232422 nb_pixel_total : 672 time to create 1 rle with old method : 0.0009708404541015625 time for calcul the mask position with numpy : 0.016609907150268555 nb_pixel_total : 703 time to create 1 rle with old method : 0.0009062290191650391 time for calcul the mask position with numpy : 0.01489567756652832 nb_pixel_total : 2691 time to create 1 rle with old method : 0.0038869380950927734 time for calcul the mask position with numpy : 0.015240907669067383 nb_pixel_total : 1328 time to create 1 rle with old method : 0.001697540283203125 time for calcul the mask position with numpy : 0.015032052993774414 nb_pixel_total : 4708 time to create 1 rle with old method : 0.0058743953704833984 time for calcul the mask position with numpy : 0.015335321426391602 nb_pixel_total : 57170 time to create 1 rle with old method : 0.06761431694030762 time for calcul the mask position with numpy : 0.014426231384277344 nb_pixel_total : 777 time to create 1 rle with old method : 0.0009770393371582031 time for calcul the mask position with numpy : 0.014333248138427734 nb_pixel_total : 1366 time to create 1 rle with old method : 0.0017507076263427734 time for calcul the mask position with numpy : 0.015000343322753906 nb_pixel_total : 38 time to create 1 rle with old method : 0.00019884109497070312 time for calcul the mask position with numpy : 0.021948575973510742 nb_pixel_total : 1544 time to create 1 rle with old method : 0.0019850730895996094 time for calcul the mask position with numpy : 0.014885663986206055 nb_pixel_total : 600 time to create 1 rle with old method : 0.0007917881011962891 time for calcul the mask position with numpy : 0.01487112045288086 nb_pixel_total : 10731 time to create 1 rle with old method : 0.01367950439453125 time for calcul the mask position with numpy : 0.014894723892211914 nb_pixel_total : 639 time to create 1 rle with old method : 0.0008399486541748047 time for calcul the mask position with numpy : 0.015410423278808594 nb_pixel_total : 680 time to create 1 rle with old method : 0.0008921623229980469 time for calcul the mask position with numpy : 0.01506185531616211 nb_pixel_total : 4058 time to create 1 rle with old method : 0.0050203800201416016 time for calcul the mask position with numpy : 0.015022993087768555 nb_pixel_total : 10 time to create 1 rle with old method : 4.553794860839844e-05 time for calcul the mask position with numpy : 0.016538381576538086 nb_pixel_total : 301 time to create 1 rle with old method : 0.0004050731658935547 time for calcul the mask position with numpy : 0.014907598495483398 nb_pixel_total : 4158 time to create 1 rle with old method : 0.005259990692138672 time for calcul the mask position with numpy : 0.01497960090637207 nb_pixel_total : 20 time to create 1 rle with old method : 7.724761962890625e-05 time for calcul the mask position with numpy : 0.01569199562072754 nb_pixel_total : 721 time to create 1 rle with old method : 0.0009291172027587891 time for calcul the mask position with numpy : 0.014892578125 nb_pixel_total : 383 time to create 1 rle with old method : 0.0005266666412353516 time for calcul the mask position with numpy : 0.014834403991699219 nb_pixel_total : 1027 time to create 1 rle with old method : 0.0013043880462646484 time for calcul the mask position with numpy : 0.015415668487548828 nb_pixel_total : 319 time to create 1 rle with old method : 0.00043201446533203125 time for calcul the mask position with numpy : 0.014865875244140625 nb_pixel_total : 263 time to create 1 rle with old method : 0.00033736228942871094 time for calcul the mask position with numpy : 0.01489710807800293 nb_pixel_total : 206 time to create 1 rle with old method : 0.0003266334533691406 time for calcul the mask position with numpy : 0.015392065048217773 nb_pixel_total : 2971 time to create 1 rle with old method : 0.0038404464721679688 time for calcul the mask position with numpy : 0.014868021011352539 nb_pixel_total : 2105 time to create 1 rle with old method : 0.002744913101196289 time for calcul the mask position with numpy : 0.014915227890014648 nb_pixel_total : 3417 time to create 1 rle with old method : 0.005122184753417969 time for calcul the mask position with numpy : 0.014902591705322266 nb_pixel_total : 1037 time to create 1 rle with old method : 0.0013394355773925781 time for calcul the mask position with numpy : 0.014853715896606445 nb_pixel_total : 487 time to create 1 rle with old method : 0.0006530284881591797 time for calcul the mask position with numpy : 0.015076875686645508 nb_pixel_total : 4800 time to create 1 rle with old method : 0.006191730499267578 time for calcul the mask position with numpy : 0.014879226684570312 nb_pixel_total : 172 time to create 1 rle with old method : 0.0002486705780029297 time for calcul the mask position with numpy : 0.014874458312988281 nb_pixel_total : 926 time to create 1 rle with old method : 0.0012025833129882812 time for calcul the mask position with numpy : 0.01572704315185547 nb_pixel_total : 229 time to create 1 rle with old method : 0.000446319580078125 time for calcul the mask position with numpy : 0.01491856575012207 nb_pixel_total : 16827 time to create 1 rle with old method : 0.022350072860717773 time for calcul the mask position with numpy : 0.014935970306396484 nb_pixel_total : 317 time to create 1 rle with old method : 0.0006034374237060547 time for calcul the mask position with numpy : 0.014934778213500977 nb_pixel_total : 82 time to create 1 rle with old method : 0.0001342296600341797 time for calcul the mask position with numpy : 0.01551675796508789 nb_pixel_total : 856 time to create 1 rle with old method : 0.0011501312255859375 time for calcul the mask position with numpy : 0.01491546630859375 nb_pixel_total : 942 time to create 1 rle with old method : 0.001214742660522461 time for calcul the mask position with numpy : 0.01486659049987793 nb_pixel_total : 3 time to create 1 rle with old method : 1.2636184692382812e-05 time for calcul the mask position with numpy : 0.015738725662231445 nb_pixel_total : 1112 time to create 1 rle with old method : 0.001428842544555664 time for calcul the mask position with numpy : 0.014872550964355469 nb_pixel_total : 254 time to create 1 rle with old method : 0.0003426074981689453 time for calcul the mask position with numpy : 0.014892578125 nb_pixel_total : 502 time to create 1 rle with old method : 0.0006673336029052734 time for calcul the mask position with numpy : 0.01575613021850586 nb_pixel_total : 793 time to create 1 rle with old method : 0.001012563705444336 create new chi : 4.2321507930755615 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.0196077823638916 batch 1 Loaded 223 chid ids of type : 4230 Number RLEs to save : 20786 TO DO : save crop sub photo not yet done ! save time : 7.666193246841431 map_output_result : {1340750929: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1340750929] Looping around the photos to save general results len do output : 1 /1340750929.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2634225') ('4234', '21027671', '1340750929', None, None, None, None, None, '2634225') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.01255035400390625 save_final save missing photos in datou_result : time spend for datou_step_exec : 21.425447702407837 time spend to save output : 0.012786865234375 total time spend for step 4 : 21.438234567642212 step5:crop_condition Wed Mar 5 04:21:23 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4230 Loading chi in step crop for list_pids : 1 ! batch 1 Loaded 223 chid ids of type : 4230 begin to crop the class : papier param for this class : {'min_score': 0.6} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 110 About to insert : list_path_to_insert length 110 new photo from crops ! About to upload 110 photos upload in portfolio : 4869462 init cache_photo without model_param we have 110 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741144886_3215133 we have uploaded 110 photos in the portfolio 4869462 time of upload the photos Elapsed time : 30.667362928390503 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.6} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 26 About to insert : list_path_to_insert length 26 new photo from crops ! About to upload 26 photos upload in portfolio : 4869462 init cache_photo without model_param we have 26 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741144917_3215133 we have uploaded 26 photos in the portfolio 4869462 time of upload the photos Elapsed time : 7.9542152881622314 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.6} filtre for class : metal hashtag_id of this class : 492628673 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741144925_3215133 we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.5050652027130127 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.6} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 16 About to insert : list_path_to_insert length 16 new photo from crops ! About to upload 16 photos upload in portfolio : 4869462 init cache_photo without model_param we have 16 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741144926_3215133 we have uploaded 16 photos in the portfolio 4869462 time of upload the photos Elapsed time : 5.45272970199585 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.6} filtre for class : autre hashtag_id of this class : 494826614 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 10 About to insert : list_path_to_insert length 10 new photo from crops ! About to upload 10 photos upload in portfolio : 4869462 init cache_photo without model_param we have 10 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741144932_3215133 we have uploaded 10 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.150954484939575 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.6} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.6} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1340750929] Looping around the photos to save general results len do output : 163 /1340930155Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930156Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930157Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930158Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930160Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930161Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930162Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930163Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930164Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930165Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930166Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930167Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930168Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930169Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930170Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930171Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930172Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930173Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930174Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930175Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930176Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930177Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930178Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930179Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930180Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930181Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930182Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930183Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930184Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930185Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930186Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930187Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930188Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930189Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930190Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930191Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930192Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930193Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930194Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930195Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930196Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930197Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930198Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930199Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930200Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930201Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930202Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930203Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930204Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930205Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930206Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930207Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930208Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930209Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930210Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930211Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930213Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930214Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930215Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930216Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930217Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930218Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930219Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930220Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930221Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930222Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930223Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930224Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930225Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930226Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930227Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930228Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930229Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930230Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930231Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930232Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930233Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930234Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930235Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930236Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930237Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930238Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930239Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930240Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930241Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930242Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930243Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930244Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930245Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930246Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930247Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930248Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930249Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930250Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930251Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930252Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930253Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930254Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930255Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930256Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930257Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930258Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930259Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930260Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930261Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930262Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930263Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930264Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930265Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930266Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930269Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930270Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930271Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930272Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930273Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930274Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930275Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930276Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930277Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930278Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930279Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930280Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930281Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930282Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930283Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930284Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930285Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930286Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930287Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930288Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930289Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930290Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930291Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930292Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930293Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930294Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930295Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930298Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930299Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930300Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930302Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930303Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930304Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930305Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930306Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930307Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930308Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930309Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930310Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930311Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930312Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930313Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930314Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930315Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930316Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930317Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930318Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930319Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930321Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930322Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930323Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930324Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340930325Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2634225') ('4234', '21027671', '1340750929', None, None, None, None, None, '2634225') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 490 time used for this insertion : 0.03621816635131836 save_final save missing photos in datou_result : time spend for datou_step_exec : 52.82137322425842 time spend to save output : 0.04055023193359375 total time spend for step 5 : 52.86192345619202 step6:thcl Wed Mar 5 04:22:15 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! we are using the classfication for only one thcl 3237 time to import caffe and check if the image exist : 0.009508371353149414 time to convert the images to numpy array : 0.03408670425415039 time to import caffe and check if the image exist : 0.006124019622802734 time to convert the images to numpy array : 0.04482746124267578 time to import caffe and check if the image exist : 0.008150339126586914 time to convert the images to numpy array : 0.05249333381652832 time to import caffe and check if the image exist : 0.009899377822875977 time to convert the images to numpy array : 0.05463361740112305 time to import caffe and check if the image exist : 0.01180720329284668 time to convert the images to numpy array : 0.05342531204223633 time to import caffe and check if the image exist : 0.016082286834716797 time to convert the images to numpy array : 0.05274033546447754 time to import caffe and check if the image exist : 0.013892650604248047 time to convert the images to numpy array : 0.05517268180847168 time to import caffe and check if the image exist : 0.01952075958251953 time to convert the images to numpy array : 0.04996204376220703 time to import caffe and check if the image exist : 0.030755996704101562 time to convert the images to numpy array : 0.04228949546813965 time to import caffe and check if the image exist : 0.018185138702392578 time to convert the images to numpy array : 0.058773040771484375 total time to convert the images to numpy array : 0.28589725494384766 list photo_ids error: [] list photo_ids correct : [1340930315, 1340930316, 1340930317, 1340930318, 1340930319, 1340930321, 1340930322, 1340930323, 1340930324, 1340930325, 1340930225, 1340930226, 1340930227, 1340930228, 1340930229, 1340930230, 1340930231, 1340930232, 1340930233, 1340930234, 1340930235, 1340930236, 1340930237, 1340930238, 1340930239, 1340930240, 1340930241, 1340930155, 1340930156, 1340930157, 1340930158, 1340930160, 1340930161, 1340930162, 1340930163, 1340930164, 1340930165, 1340930166, 1340930167, 1340930168, 1340930169, 1340930170, 1340930171, 1340930172, 1340930259, 1340930260, 1340930261, 1340930262, 1340930263, 1340930264, 1340930265, 1340930266, 1340930269, 1340930270, 1340930271, 1340930272, 1340930273, 1340930274, 1340930275, 1340930276, 1340930277, 1340930278, 1340930279, 1340930280, 1340930281, 1340930282, 1340930283, 1340930284, 1340930285, 1340930286, 1340930287, 1340930288, 1340930289, 1340930290, 1340930291, 1340930292, 1340930293, 1340930294, 1340930190, 1340930191, 1340930192, 1340930193, 1340930194, 1340930195, 1340930196, 1340930197, 1340930198, 1340930199, 1340930200, 1340930201, 1340930202, 1340930203, 1340930204, 1340930205, 1340930206, 1340930207, 1340930208, 1340930209, 1340930210, 1340930211, 1340930213, 1340930214, 1340930215, 1340930216, 1340930217, 1340930218, 1340930219, 1340930220, 1340930221, 1340930222, 1340930223, 1340930224, 1340930242, 1340930243, 1340930244, 1340930245, 1340930246, 1340930247, 1340930248, 1340930249, 1340930250, 1340930251, 1340930252, 1340930253, 1340930254, 1340930255, 1340930256, 1340930257, 1340930258, 1340930173, 1340930174, 1340930175, 1340930176, 1340930177, 1340930178, 1340930179, 1340930180, 1340930181, 1340930182, 1340930183, 1340930184, 1340930185, 1340930186, 1340930187, 1340930188, 1340930189, 1340930295, 1340930298, 1340930299, 1340930300, 1340930302, 1340930303, 1340930304, 1340930305, 1340930306, 1340930307, 1340930308, 1340930309, 1340930310, 1340930311, 1340930312, 1340930313, 1340930314] number of photos to traite : 163 try to delete the photos incorrect in DB tagging for thcl : 3237 To do loadFromThcl(), then load ParamDescType : thcl3237 thcls : [{'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'}] thcl {'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'} Update svm_hashtag_type_desc : 5561 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) To loadFromThcl() : net_5561 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10593 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_rubbia_refus_2500 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_rubbia_refus_2500 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_rubbia_refus_2500 /data/models_weight/learn_rubbia_refus_2500/caffemodel size_local : 94358479 size in s3 : 94358479 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:16 caffemodel already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt size_local : 32544 size in s3 : 32544 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:15 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:09:52 mean.npy already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/synset_words.txt size_local : 334 size in s3 : 334 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:10:06 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt caffemodel_filename : /data/models_weight/learn_rubbia_refus_2500/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10374 max_wait_temp : 1 max_wait : 0 dict_keys(['prob', 'pool5']) time used to do the prepocess of the images : 0.8272078037261963 time used to do the prediction : 0.893841028213501 save descriptor for thcl : 3237 time to traite the descriptors : 0.9354372024536133 Catched exception ! Connect or reconnect ! storage_type for insertDescriptorsMulti : 3 To insert : 1340930315 To insert : 1340930316 To insert : 1340930317 To insert : 1340930318 To insert : 1340930319 To insert : 1340930321 To insert : 1340930322 To insert : 1340930323 To insert : 1340930324 To insert : 1340930325 To insert : 1340930225 To insert : 1340930226 To insert : 1340930227 To insert : 1340930228 To insert : 1340930229 To insert : 1340930230 To insert : 1340930231 To insert : 1340930232 To insert : 1340930233 To insert : 1340930234 To insert : 1340930235 To insert : 1340930236 To insert : 1340930237 To insert : 1340930238 To insert : 1340930239 To insert : 1340930240 To insert : 1340930241 To insert : 1340930155 To insert : 1340930156 To insert : 1340930157 To insert : 1340930158 To insert : 1340930160 To insert : 1340930161 To insert : 1340930162 To insert : 1340930163 To insert : 1340930164 To insert : 1340930165 To insert : 1340930166 To insert : 1340930167 To insert : 1340930168 To insert : 1340930169 To insert : 1340930170 To insert : 1340930171 To insert : 1340930172 To insert : 1340930259 To insert : 1340930260 To insert : 1340930261 To insert : 1340930262 To insert : 1340930263 To insert : 1340930264 To insert : 1340930265 To insert : 1340930266 To insert : 1340930269 To insert : 1340930270 To insert : 1340930271 To insert : 1340930272 To insert : 1340930273 To insert : 1340930274 To insert : 1340930275 To insert : 1340930276 To insert : 1340930277 To insert : 1340930278 To insert : 1340930279 To insert : 1340930280 To insert : 1340930281 To insert : 1340930282 To insert : 1340930283 To insert : 1340930284 To insert : 1340930285 To insert : 1340930286 To insert : 1340930287 To insert : 1340930288 To insert : 1340930289 To insert : 1340930290 To insert : 1340930291 To insert : 1340930292 To insert : 1340930293 To insert : 1340930294 To insert : 1340930190 To insert : 1340930191 To insert : 1340930192 To insert : 1340930193 To insert : 1340930194 To insert : 1340930195 To insert : 1340930196 To insert : 1340930197 To insert : 1340930198 To insert : 1340930199 To insert : 1340930200 To insert : 1340930201 To insert : 1340930202 To insert : 1340930203 To insert : 1340930204 To insert : 1340930205 To insert : 1340930206 To insert : 1340930207 To insert : 1340930208 To insert : 1340930209 To insert : 1340930210 To insert : 1340930211 To insert : 1340930213 To insert : 1340930214 To insert : 1340930215 To insert : 1340930216 To insert : 1340930217 To insert : 1340930218 To insert : 1340930219 To insert : 1340930220 To insert : 1340930221 To insert : 1340930222 To insert : 1340930223 To insert : 1340930224 To insert : 1340930242 To insert : 1340930243 To insert : 1340930244 To insert : 1340930245 To insert : 1340930246 To insert : 1340930247 To insert : 1340930248 To insert : 1340930249 To insert : 1340930250 To insert : 1340930251 To insert : 1340930252 To insert : 1340930253 To insert : 1340930254 To insert : 1340930255 To insert : 1340930256 To insert : 1340930257 To insert : 1340930258 To insert : 1340930173 To insert : 1340930174 To insert : 1340930175 To insert : 1340930176 To insert : 1340930177 To insert : 1340930178 To insert : 1340930179 To insert : 1340930180 To insert : 1340930181 To insert : 1340930182 To insert : 1340930183 To insert : 1340930184 To insert : 1340930185 To insert : 1340930186 To insert : 1340930187 To insert : 1340930188 To insert : 1340930189 To insert : 1340930295 To insert : 1340930298 To insert : 1340930299 To insert : 1340930300 To insert : 1340930302 To insert : 1340930303 To insert : 1340930304 To insert : 1340930305 To insert : 1340930306 To insert : 1340930307 To insert : 1340930308 To insert : 1340930309 To insert : 1340930310 To insert : 1340930311 To insert : 1340930312 To insert : 1340930313 To insert : 1340930314 time to insert the descriptors : 32.246519327163696 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos SAVE THCL : begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 244 time used for this insertion : 0.4285247325897217 save missing photos in datou_result : time spend for datou_step_exec : 39.19862723350525 time spend to save output : 0.47759437561035156 total time spend for step 6 : 39.6762216091156 step7:ventilate_hashtags_in_portfolio Wed Mar 5 04:22:55 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 21027671 get user id for portfolio 21027671 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027671 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('environnement','carton','pehd','Film_plastique','autre','metal','refus','papier','flou','Carton','PET_clair','pet_clair','mal_croppe','pet_fonce','PEHD','PET_fonce','Papier','Tetrapak')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027671 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('environnement','carton','pehd','Film_plastique','autre','metal','refus','papier','flou','Carton','PET_clair','pet_clair','mal_croppe','pet_fonce','PEHD','PET_fonce','Papier','Tetrapak')) AND mptpi.`min_score`=0.6 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027671 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('environnement','carton','pehd','Film_plastique','autre','metal','refus','papier','flou','Carton','PET_clair','pet_clair','mal_croppe','pet_fonce','PEHD','PET_fonce','Papier','Tetrapak')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027671 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('environnement','carton','pehd','Film_plastique','autre','metal','refus','papier','flou','Carton','PET_clair','pet_clair','mal_croppe','pet_fonce','PEHD','PET_fonce','Papier','Tetrapak')) AND mptpi.`min_score`=0.6 To do lien utilise dans velours : https://www.fotonower.com/velours/21042618,21042627,21042632,21042621,21042622,21042623,21042624,21042634,21042626,21042629,21042630,21042633,21042635?tags=environnement,carton,pehd,film_plastique,autre,metal,refus,papier,flou,pet_clair,mal_croppe,pet_fonce,tetrapak&datou_id_consolidate=4235&port_consolidate=21027671 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1340750929] Looping around the photos to save general results len do output : 1 /21027671. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2634225') ('4234', '21027671', '1340750929', None, None, None, None, None, '2634225') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 2 time used for this insertion : 0.012022733688354492 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.602367401123047 time spend to save output : 0.012279748916625977 total time spend for step 7 : 2.614647150039673 step8:final Wed Mar 5 04:22:58 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1340750929: ('0.8040172653862341',)} new output for save of step final : {1340750929: ('0.8040172653862341',)} [1340750929] Looping around the photos to save general results len do output : 1 /1340750929.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2634225') ('4234', '21027671', '1340750929', None, None, None, None, None, '2634225') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.015786170959472656 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.1147909164428711 time spend to save output : 0.01604294776916504 total time spend for step 8 : 0.13083386421203613 step9:velours_tree Wed Mar 5 04:22:58 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure list_portfolios : 21042618,21042627,21042632,21042621,21042622,21042623,21042624,21042634,21042626,21042629,21042630,21042633,21042635 photo desc type : 5561 - Retrieving photos to tag... query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21042618 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21042627 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21042632 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21042621 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21042622 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21042623 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21042624 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21042634 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21042626 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21042629 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21042630 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21042633 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21042635 ORDER BY ph.size desc - Loading descriptors... Size : 2048 len(descriptors) : 163 Compute structured hierarchical clustering... ward : AgglomerativeClustering(n_clusters=6) ward.labels_ : [0 0 0 1 1 4 0 4 0 2 2 4 2 2 2 5 3 2 4 0 0 4 4 4 4 3 3 3 2 5 3 0 0 4 4 0 0 0 0 0 3 2 0 1 1 0 2 2 2 4 2 3 0 3 0 0 0 0 4 0 0 0 3 1 2 5 0 0 0 0 0 0 5 2 0 1 0 2 0 0 0 0 0 0 1 0 3 0 3 3 5 0 5 0 2 1 1 0 0 4 1 0 3 3 4 5 1 0 4 1 1 1 2 0 1 0 1 1 5 0 1 2 1 0 1 0 1 2 0 0 0 0 5 0 0 0 4 3 0 2 2 1 3 0 2 1 3 2 0 0 0 5 5 5 3 5 4 0 2 3 1 1 0] Elapsed time: 0.0298917293548584 graph_id used : 81503 - Beta version, working pretty good on 11-5-16 ! https://www.fotonower.com/velours/21042618,21042627,21042632,21042621,21042622,21042623,21042624,21042634,21042626,21042629,21042630,21042633,21042635?tags=environnement,carton,pehd,film_plastique,autre,metal,refus,papier,flou,pet_clair,mal_croppe,pet_fonce,tetrapak&datou_id_consolidate=4235&port_consolidate=21027671&tree_id=81503 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : velours_tree we use saveGeneral [1340750929] Looping around the photos to save general results len do output : 1 /21027671Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2634225') ('4234', '21027671', '1340750929', None, None, None, None, None, '2634225') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 2 time used for this insertion : 0.015250921249389648 save_final save missing photos in datou_result : time spend for datou_step_exec : 11.354458808898926 time spend to save output : 0.01548624038696289 total time spend for step 9 : 11.369945049285889 step10:send_mail_cod Wed Mar 5 04:23:09 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 complete output_args for input 2 complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Qualipapia_P21027671_05-03-2025_04_23_09.pdf 21042609 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette210426091741144989 21042614 imagette210426141741144990 21042603 imagette210426031741144990 21042604 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette210426041741144990 21042605 change filename to text .imagette210426051741144991 21042606 imagette210426061741144991 21042616 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette210426161741144991 21042608 imagette210426081741144992 21042611 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette210426111741144992 21042612 imagette210426121741144993 21042615 imagette210426151741144993 21042617 imagette210426171741144993 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=21027671 and hashtag_type = 4230 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/21042618,21042627,21042632,21042621,21042622,21042623,21042624,21042634,21042626,21042629,21042630,21042633,21042635?tags=environnement,carton,pehd,film_plastique,autre,metal,refus,papier,flou,pet_clair,mal_croppe,pet_fonce,tetrapak&datou_id_consolidate=4235&port_consolidate=21027671&tree_id=81503 your option no_mail is active, we will not send the real mail to your client args[1340750929] : ((1340750929, -6.494564113351858, 492609224), (1340750929, 0.3571968249559575, 2107752395), '0.8040172653862341') We are sending mail with results at cod@fotonower.com refus_total : 0.8040172653862341 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21027671 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027671_05-03-2025_04_23_09.pdf results_Qualipapia_P21027671_05-03-2025_04_23_09.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027671_05-03-2025_04_23_09.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4234','21027671','results_Qualipapia_P21027671_05-03-2025_04_23_09.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027671_05-03-2025_04_23_09.pdf','pdf','','0.14','0.8040172653862341') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1340750929] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2634225') ('4234', '21027671', '1340750929', None, None, None, None, None, '2634225') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 1 time used for this insertion : 0.27863550186157227 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.760587215423584 time spend to save output : 0.27880430221557617 total time spend for step 10 : 5.03939151763916 step11:split_time_score Wed Mar 5 04:23:14 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 3442, 'mtr_user_id': 31, 'name': 'classifieur_2camions_valcor_021122_v1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'deux_camions,camion_droite,camion_gauche,pas_de_camion', 'svm_portfolios_learning': '7659379,7659034,7657685,7657114', 'photo_hashtag_type': 4458, 'photo_desc_type': 5723, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107760533,2107760534,2107760535,2107760536'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('20', 1),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 04032025 21027671 Nombre de photos uploadées : 1 / 23040 (0%) 04032025 21027671 Nombre de photos taguées (types de déchets): 0 / 1 (0%) 04032025 21027671 Nombre de photos taguées (volume) : 0 / 1 (0%) elapsed_time : load_data_split_time_score 2.384185791015625e-06 elapsed_time : order_list_meta_photo_and_scores 5.7220458984375e-06 ? elapsed_time : fill_and_build_computed_from_old_data 0.00014853477478027344 elapsed_time : insert_dashboard_record_day_entry 0.028176069259643555 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027616 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027617 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027618 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027619 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027668 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027669 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027670 order by id desc limit 1 Qualite : 0.8040172653862341 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027671_05-03-2025_04_23_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027671 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027671 AND mptpi.`type`=4230 To do Qualite : 0.8661318078018976 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027672_05-03-2025_01_34_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027672 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027672 AND mptpi.`type`=4230 To do Qualite : 0.7959813288093653 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027673_05-03-2025_01_49_19.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027673 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027673 AND mptpi.`type`=4230 To do Qualite : 0.834470763448575 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027674_05-03-2025_03_48_54.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027674 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027674 AND mptpi.`type`=4230 To do Qualite : 0.851646311193362 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027675_05-03-2025_00_48_42.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027675 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027675 AND mptpi.`type`=4230 To do Qualite : 0.7966271289732663 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027676_05-03-2025_00_36_58.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027676 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027676 AND mptpi.`type`=4230 To do Qualite : 0.799150804878469 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21009642_04-03-2025_21_26_08.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21009642 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21009642 AND mptpi.`type`=4230 To do Qualite : 0.7978167118520791 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027677_04-03-2025_23_49_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027677 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027677 AND mptpi.`type`=4230 To do Qualite : 0.8168670033242211 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027678_04-03-2025_22_34_26.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027678 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027678 AND mptpi.`type`=4230 To do Qualite : 0.8002751529547594 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027679_04-03-2025_22_48_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027679 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027679 AND mptpi.`type`=4230 To do Qualite : 0.7799330629939157 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027680_04-03-2025_23_32_32.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027680 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027680 AND mptpi.`type`=4230 To do Qualite : 0.7920543799595827 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027681_04-03-2025_22_02_31.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027681 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027681 AND mptpi.`type`=4230 To do Qualite : 0.7743571987821605 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027682_04-03-2025_22_18_19.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027682 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027682 AND mptpi.`type`=4230 To do Qualite : 0.7757546150050475 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21027683_04-03-2025_22_24_43.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21027683 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21027683 AND mptpi.`type`=4230 To do Qualite : 0.7784556948713458 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21030444_04-03-2025_23_21_47.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21030444 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21030444 AND mptpi.`type`=4230 To do Qualite : 0.7581416845186528 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P21033064_05-03-2025_00_19_39.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21033064 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21033064 AND mptpi.`type`=4230 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'04032025': {'nb_upload': 1, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1340750929] Looping around the photos to save general results len do output : 1 /21027671Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2634225') ('4234', '21027671', '1340750929', None, None, None, None, None, '2634225') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 2 time used for this insertion : 0.07894444465637207 save_final save missing photos in datou_result : time spend for datou_step_exec : 7.170766353607178 time spend to save output : 0.07907438278198242 total time spend for step 11 : 7.24984073638916 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 1 len(input) + len(total_photo_id_missing) : 1 set_done_treatment 38.92user 23.86system 2:57.14elapsed 35%CPU (0avgtext+0avgdata 2499452maxresident)k 1155976inputs+17160outputs (3171major+3081667minor)pagefaults 0swaps