python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 3459 -l 10 ' -s datou_current_3459 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1438157 load datou : 3459 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2868 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5288 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 we have a portfolio with more photos than limit : 20>10 please execute split_portfolio.py -i 21291428 -l 10 size over we load limit photo not treated list_input_json: [] Current got : datou_id : 3459, datou_cur_ids : ['2658518'] with mtr_portfolio_ids : ['21291428'] and first list_photo_ids : [] new path : /proc/1438157/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, thcl, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 10 ; length of list_pids : 10 ; length of list_args : 10 time to download the photos : 1.6499502658843994 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Tue Mar 11 15:30:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10151 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-03-11 15:30:33.622452: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-03-11 15:30:33.655176: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-03-11 15:30:33.656972: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7feb20000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-03-11 15:30:33.657021: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-03-11 15:30:33.661415: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-03-11 15:30:33.946651: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5db3cb0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-03-11 15:30:33.946697: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-03-11 15:30:33.948190: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-11 15:30:33.948649: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-11 15:30:33.950824: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-11 15:30:33.953119: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-11 15:30:33.953453: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-11 15:30:33.956749: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-11 15:30:33.958243: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-11 15:30:33.964133: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-11 15:30:33.965577: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-11 15:30:33.965640: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-11 15:30:33.966431: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-11 15:30:33.966447: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-11 15:30:33.966456: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-11 15:30:33.967832: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9399 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-03-11 15:30:34.293977: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-11 15:30:34.294170: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-11 15:30:34.294215: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-11 15:30:34.294251: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-11 15:30:34.294284: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-11 15:30:34.294316: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-11 15:30:34.294351: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-11 15:30:34.294380: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-11 15:30:34.296121: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-11 15:30:34.297906: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-11 15:30:34.297977: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-11 15:30:34.298006: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-11 15:30:34.298030: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-11 15:30:34.298054: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-11 15:30:34.298078: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-11 15:30:34.298104: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-11 15:30:34.298130: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-11 15:30:34.299751: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-11 15:30:34.299809: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-11 15:30:34.299826: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-11 15:30:34.299839: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-11 15:30:34.301533: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9399 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-03-11 15:30:45.092064: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-11 15:30:45.364399: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 1 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 23 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 23 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 24 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 17 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 16 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 19 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 15 Detection mask done ! Trying to reset tf kernel 1438756 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1085 tf kernel not reseted sub process len(results) : 10 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 10 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 6374 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.35909533500671387 nb_pixel_total : 3925724 time to create 1 rle with new method : 1.2027380466461182 length of segment : 4569 time for calcul the mask position with numpy : 0.001987934112548828 nb_pixel_total : 38069 time to create 1 rle with old method : 0.0471348762512207 length of segment : 350 time for calcul the mask position with numpy : 0.017784595489501953 nb_pixel_total : 847567 time to create 1 rle with new method : 0.0622251033782959 length of segment : 1824 time for calcul the mask position with numpy : 0.8740577697753906 nb_pixel_total : 5203424 time to create 1 rle with new method : 0.45776963233947754 length of segment : 3796 time for calcul the mask position with numpy : 0.004576683044433594 nb_pixel_total : 150965 time to create 1 rle with new method : 0.007949590682983398 length of segment : 917 time for calcul the mask position with numpy : 0.00027680397033691406 nb_pixel_total : 4833 time to create 1 rle with old method : 0.00600123405456543 length of segment : 81 time for calcul the mask position with numpy : 0.001581430435180664 nb_pixel_total : 45669 time to create 1 rle with old method : 0.05249452590942383 length of segment : 184 time for calcul the mask position with numpy : 0.00028705596923828125 nb_pixel_total : 6066 time to create 1 rle with old method : 0.007874488830566406 length of segment : 71 time for calcul the mask position with numpy : 0.0021088123321533203 nb_pixel_total : 37920 time to create 1 rle with old method : 0.043653249740600586 length of segment : 455 time for calcul the mask position with numpy : 0.0004992485046386719 nb_pixel_total : 13425 time to create 1 rle with old method : 0.015942096710205078 length of segment : 121 time for calcul the mask position with numpy : 0.0072460174560546875 nb_pixel_total : 235420 time to create 1 rle with new method : 0.012082338333129883 length of segment : 925 time for calcul the mask position with numpy : 0.00011467933654785156 nb_pixel_total : 2794 time to create 1 rle with old method : 0.0034117698669433594 length of segment : 98 time for calcul the mask position with numpy : 0.005757808685302734 nb_pixel_total : 283026 time to create 1 rle with new method : 0.014120101928710938 length of segment : 863 time for calcul the mask position with numpy : 0.0018792152404785156 nb_pixel_total : 52162 time to create 1 rle with old method : 0.05884361267089844 length of segment : 539 time for calcul the mask position with numpy : 0.0004520416259765625 nb_pixel_total : 10248 time to create 1 rle with old method : 0.012084245681762695 length of segment : 107 time for calcul the mask position with numpy : 0.0009436607360839844 nb_pixel_total : 23672 time to create 1 rle with old method : 0.026873350143432617 length of segment : 283 time for calcul the mask position with numpy : 0.00032329559326171875 nb_pixel_total : 5961 time to create 1 rle with old method : 0.007355928421020508 length of segment : 67 time for calcul the mask position with numpy : 0.0005838871002197266 nb_pixel_total : 15123 time to create 1 rle with old method : 0.018121957778930664 length of segment : 154 time for calcul the mask position with numpy : 0.001210927963256836 nb_pixel_total : 42831 time to create 1 rle with old method : 0.04884696006774902 length of segment : 278 time for calcul the mask position with numpy : 0.002737760543823242 nb_pixel_total : 74725 time to create 1 rle with old method : 0.08698749542236328 length of segment : 350 time for calcul the mask position with numpy : 0.0005581378936767578 nb_pixel_total : 26462 time to create 1 rle with old method : 0.03295493125915527 length of segment : 211 time for calcul the mask position with numpy : 0.00021409988403320312 nb_pixel_total : 2163 time to create 1 rle with old method : 0.002695322036743164 length of segment : 68 time for calcul the mask position with numpy : 0.0027112960815429688 nb_pixel_total : 54676 time to create 1 rle with old method : 0.06189274787902832 length of segment : 631 time for calcul the mask position with numpy : 0.0010325908660888672 nb_pixel_total : 24367 time to create 1 rle with old method : 0.02805948257446289 length of segment : 255 time for calcul the mask position with numpy : 0.002862215042114258 nb_pixel_total : 91803 time to create 1 rle with old method : 0.10329890251159668 length of segment : 472 time for calcul the mask position with numpy : 0.004013538360595703 nb_pixel_total : 137131 time to create 1 rle with old method : 0.152449369430542 length of segment : 404 time for calcul the mask position with numpy : 0.0010344982147216797 nb_pixel_total : 25772 time to create 1 rle with old method : 0.030316591262817383 length of segment : 322 time for calcul the mask position with numpy : 0.0007176399230957031 nb_pixel_total : 16398 time to create 1 rle with old method : 0.01948404312133789 length of segment : 157 time for calcul the mask position with numpy : 0.0018284320831298828 nb_pixel_total : 29031 time to create 1 rle with old method : 0.03448128700256348 length of segment : 309 time for calcul the mask position with numpy : 0.0021398067474365234 nb_pixel_total : 50554 time to create 1 rle with old method : 0.060919761657714844 length of segment : 480 time for calcul the mask position with numpy : 0.0008947849273681641 nb_pixel_total : 23769 time to create 1 rle with old method : 0.029020309448242188 length of segment : 189 time for calcul the mask position with numpy : 0.000286102294921875 nb_pixel_total : 14790 time to create 1 rle with old method : 0.017134428024291992 length of segment : 139 time for calcul the mask position with numpy : 0.0002872943878173828 nb_pixel_total : 4823 time to create 1 rle with old method : 0.006120443344116211 length of segment : 75 time for calcul the mask position with numpy : 0.002259969711303711 nb_pixel_total : 51758 time to create 1 rle with old method : 0.05966472625732422 length of segment : 257 time for calcul the mask position with numpy : 0.001961946487426758 nb_pixel_total : 48107 time to create 1 rle with old method : 0.05906176567077637 length of segment : 364 time for calcul the mask position with numpy : 0.0016639232635498047 nb_pixel_total : 55866 time to create 1 rle with old method : 0.06312942504882812 length of segment : 408 time for calcul the mask position with numpy : 0.00028967857360839844 nb_pixel_total : 3241 time to create 1 rle with old method : 0.01562976837158203 length of segment : 74 time for calcul the mask position with numpy : 0.01805877685546875 nb_pixel_total : 213661 time to create 1 rle with new method : 0.07749390602111816 length of segment : 869 time for calcul the mask position with numpy : 0.002375364303588867 nb_pixel_total : 52253 time to create 1 rle with old method : 0.06099128723144531 length of segment : 287 time for calcul the mask position with numpy : 0.00030350685119628906 nb_pixel_total : 19244 time to create 1 rle with old method : 0.022125244140625 length of segment : 212 time for calcul the mask position with numpy : 0.0058710575103759766 nb_pixel_total : 216530 time to create 1 rle with new method : 0.017537355422973633 length of segment : 893 time for calcul the mask position with numpy : 0.003844022750854492 nb_pixel_total : 76833 time to create 1 rle with old method : 0.08605051040649414 length of segment : 594 time for calcul the mask position with numpy : 0.002378702163696289 nb_pixel_total : 13155 time to create 1 rle with old method : 0.015283346176147461 length of segment : 140 time for calcul the mask position with numpy : 0.004301548004150391 nb_pixel_total : 178995 time to create 1 rle with new method : 0.013514995574951172 length of segment : 781 time for calcul the mask position with numpy : 0.0005621910095214844 nb_pixel_total : 5706 time to create 1 rle with old method : 0.0068988800048828125 length of segment : 101 time for calcul the mask position with numpy : 0.00012755393981933594 nb_pixel_total : 888 time to create 1 rle with old method : 0.0010895729064941406 length of segment : 54 time for calcul the mask position with numpy : 0.004088163375854492 nb_pixel_total : 120907 time to create 1 rle with old method : 0.13907098770141602 length of segment : 546 time for calcul the mask position with numpy : 0.0017771720886230469 nb_pixel_total : 53978 time to create 1 rle with old method : 0.06030154228210449 length of segment : 317 time for calcul the mask position with numpy : 0.003416299819946289 nb_pixel_total : 141601 time to create 1 rle with old method : 0.15837764739990234 length of segment : 415 time for calcul the mask position with numpy : 0.005570411682128906 nb_pixel_total : 174166 time to create 1 rle with new method : 0.007802009582519531 length of segment : 587 time for calcul the mask position with numpy : 0.00026869773864746094 nb_pixel_total : 3766 time to create 1 rle with old method : 0.004515647888183594 length of segment : 66 time for calcul the mask position with numpy : 0.0019490718841552734 nb_pixel_total : 56000 time to create 1 rle with old method : 0.06187558174133301 length of segment : 430 time for calcul the mask position with numpy : 0.0012679100036621094 nb_pixel_total : 14003 time to create 1 rle with old method : 0.015323638916015625 length of segment : 300 time for calcul the mask position with numpy : 0.0007331371307373047 nb_pixel_total : 18168 time to create 1 rle with old method : 0.02652573585510254 length of segment : 135 time for calcul the mask position with numpy : 0.0053920745849609375 nb_pixel_total : 161144 time to create 1 rle with new method : 0.011770963668823242 length of segment : 915 time for calcul the mask position with numpy : 0.0002014636993408203 nb_pixel_total : 3264 time to create 1 rle with old method : 0.003999233245849609 length of segment : 74 time for calcul the mask position with numpy : 0.0002989768981933594 nb_pixel_total : 10334 time to create 1 rle with old method : 0.012152671813964844 length of segment : 109 time for calcul the mask position with numpy : 0.00031304359436035156 nb_pixel_total : 5805 time to create 1 rle with old method : 0.006373405456542969 length of segment : 121 time for calcul the mask position with numpy : 0.0026574134826660156 nb_pixel_total : 99976 time to create 1 rle with old method : 0.10826945304870605 length of segment : 362 time for calcul the mask position with numpy : 0.0005087852478027344 nb_pixel_total : 17522 time to create 1 rle with old method : 0.020984649658203125 length of segment : 178 time for calcul the mask position with numpy : 0.0017414093017578125 nb_pixel_total : 41565 time to create 1 rle with old method : 0.04898524284362793 length of segment : 392 time for calcul the mask position with numpy : 0.0024261474609375 nb_pixel_total : 90307 time to create 1 rle with old method : 0.1003570556640625 length of segment : 981 time for calcul the mask position with numpy : 0.005448579788208008 nb_pixel_total : 177179 time to create 1 rle with new method : 0.008822441101074219 length of segment : 862 time for calcul the mask position with numpy : 0.004346132278442383 nb_pixel_total : 180222 time to create 1 rle with new method : 0.013062477111816406 length of segment : 1127 time for calcul the mask position with numpy : 0.00041794776916503906 nb_pixel_total : 16760 time to create 1 rle with old method : 0.019194841384887695 length of segment : 169 time for calcul the mask position with numpy : 0.0025472640991210938 nb_pixel_total : 113849 time to create 1 rle with old method : 0.13659954071044922 length of segment : 910 time for calcul the mask position with numpy : 0.0005877017974853516 nb_pixel_total : 10559 time to create 1 rle with old method : 0.01763463020324707 length of segment : 196 time for calcul the mask position with numpy : 0.0006852149963378906 nb_pixel_total : 10979 time to create 1 rle with old method : 0.013399362564086914 length of segment : 147 time for calcul the mask position with numpy : 0.008385658264160156 nb_pixel_total : 245722 time to create 1 rle with new method : 0.013950347900390625 length of segment : 635 time for calcul the mask position with numpy : 0.001657247543334961 nb_pixel_total : 50988 time to create 1 rle with old method : 0.0590054988861084 length of segment : 496 time for calcul the mask position with numpy : 0.0001761913299560547 nb_pixel_total : 2329 time to create 1 rle with old method : 0.002930164337158203 length of segment : 67 time for calcul the mask position with numpy : 0.0015022754669189453 nb_pixel_total : 99130 time to create 1 rle with old method : 0.11372995376586914 length of segment : 353 time for calcul the mask position with numpy : 0.0030891895294189453 nb_pixel_total : 129854 time to create 1 rle with old method : 0.1517801284790039 length of segment : 781 time for calcul the mask position with numpy : 0.001249074935913086 nb_pixel_total : 35559 time to create 1 rle with old method : 0.05040740966796875 length of segment : 322 time for calcul the mask position with numpy : 0.015874862670898438 nb_pixel_total : 697220 time to create 1 rle with new method : 0.3347318172454834 length of segment : 815 time for calcul the mask position with numpy : 0.0011150836944580078 nb_pixel_total : 41013 time to create 1 rle with old method : 0.046066999435424805 length of segment : 272 time for calcul the mask position with numpy : 0.0004639625549316406 nb_pixel_total : 15023 time to create 1 rle with old method : 0.01766180992126465 length of segment : 117 time for calcul the mask position with numpy : 0.0005834102630615234 nb_pixel_total : 16097 time to create 1 rle with old method : 0.021663427352905273 length of segment : 176 time for calcul the mask position with numpy : 0.0013496875762939453 nb_pixel_total : 58260 time to create 1 rle with old method : 0.07274055480957031 length of segment : 237 time for calcul the mask position with numpy : 0.002236604690551758 nb_pixel_total : 53953 time to create 1 rle with old method : 0.07908010482788086 length of segment : 358 time for calcul the mask position with numpy : 0.002050638198852539 nb_pixel_total : 65665 time to create 1 rle with old method : 0.07851815223693848 length of segment : 246 time for calcul the mask position with numpy : 0.0006473064422607422 nb_pixel_total : 17600 time to create 1 rle with old method : 0.02174687385559082 length of segment : 200 time for calcul the mask position with numpy : 0.003096342086791992 nb_pixel_total : 119805 time to create 1 rle with old method : 0.13704919815063477 length of segment : 437 time for calcul the mask position with numpy : 0.003090381622314453 nb_pixel_total : 142261 time to create 1 rle with old method : 0.15723562240600586 length of segment : 423 time for calcul the mask position with numpy : 0.005228519439697266 nb_pixel_total : 255776 time to create 1 rle with new method : 0.00777435302734375 length of segment : 703 time for calcul the mask position with numpy : 0.0011861324310302734 nb_pixel_total : 17847 time to create 1 rle with old method : 0.030873775482177734 length of segment : 135 time for calcul the mask position with numpy : 0.00027441978454589844 nb_pixel_total : 2644 time to create 1 rle with old method : 0.003139495849609375 length of segment : 141 time for calcul the mask position with numpy : 0.0005967617034912109 nb_pixel_total : 15858 time to create 1 rle with old method : 0.01805710792541504 length of segment : 211 time for calcul the mask position with numpy : 0.001064300537109375 nb_pixel_total : 8428 time to create 1 rle with old method : 0.010096073150634766 length of segment : 305 time for calcul the mask position with numpy : 0.0010929107666015625 nb_pixel_total : 45284 time to create 1 rle with old method : 0.05229353904724121 length of segment : 198 time for calcul the mask position with numpy : 0.0003058910369873047 nb_pixel_total : 8351 time to create 1 rle with old method : 0.010023832321166992 length of segment : 147 time for calcul the mask position with numpy : 0.0002181529998779297 nb_pixel_total : 6342 time to create 1 rle with old method : 0.007926464080810547 length of segment : 66 time for calcul the mask position with numpy : 0.0002574920654296875 nb_pixel_total : 6293 time to create 1 rle with old method : 0.007718086242675781 length of segment : 79 time for calcul the mask position with numpy : 0.002706289291381836 nb_pixel_total : 80976 time to create 1 rle with old method : 0.08992362022399902 length of segment : 468 time for calcul the mask position with numpy : 0.000225067138671875 nb_pixel_total : 2310 time to create 1 rle with old method : 0.0028629302978515625 length of segment : 49 time for calcul the mask position with numpy : 0.0007512569427490234 nb_pixel_total : 20241 time to create 1 rle with old method : 0.02405834197998047 length of segment : 206 time for calcul the mask position with numpy : 0.0005385875701904297 nb_pixel_total : 14990 time to create 1 rle with old method : 0.017870187759399414 length of segment : 133 time for calcul the mask position with numpy : 0.0006499290466308594 nb_pixel_total : 13725 time to create 1 rle with old method : 0.016324520111083984 length of segment : 179 time for calcul the mask position with numpy : 0.00046753883361816406 nb_pixel_total : 12609 time to create 1 rle with old method : 0.01498866081237793 length of segment : 143 time for calcul the mask position with numpy : 0.0001494884490966797 nb_pixel_total : 1394 time to create 1 rle with old method : 0.001760244369506836 length of segment : 56 time for calcul the mask position with numpy : 0.0007326602935791016 nb_pixel_total : 29732 time to create 1 rle with old method : 0.03612709045410156 length of segment : 177 time for calcul the mask position with numpy : 0.0024259090423583984 nb_pixel_total : 104221 time to create 1 rle with old method : 0.11724519729614258 length of segment : 404 time for calcul the mask position with numpy : 0.0001347064971923828 nb_pixel_total : 1097 time to create 1 rle with old method : 0.0014150142669677734 length of segment : 49 time for calcul the mask position with numpy : 0.0016758441925048828 nb_pixel_total : 74175 time to create 1 rle with old method : 0.08659696578979492 length of segment : 345 time for calcul the mask position with numpy : 0.00033211708068847656 nb_pixel_total : 5145 time to create 1 rle with old method : 0.006503105163574219 length of segment : 79 time for calcul the mask position with numpy : 0.0005271434783935547 nb_pixel_total : 11219 time to create 1 rle with old method : 0.013815164566040039 length of segment : 159 time for calcul the mask position with numpy : 0.0005438327789306641 nb_pixel_total : 10038 time to create 1 rle with old method : 0.012001514434814453 length of segment : 190 time for calcul the mask position with numpy : 0.008109807968139648 nb_pixel_total : 272720 time to create 1 rle with new method : 0.01242518424987793 length of segment : 608 time for calcul the mask position with numpy : 0.0002589225769042969 nb_pixel_total : 3104 time to create 1 rle with old method : 0.00394439697265625 length of segment : 105 time for calcul the mask position with numpy : 0.0018031597137451172 nb_pixel_total : 46941 time to create 1 rle with old method : 0.05713462829589844 length of segment : 397 time for calcul the mask position with numpy : 0.0003466606140136719 nb_pixel_total : 7183 time to create 1 rle with old method : 0.009386301040649414 length of segment : 91 time for calcul the mask position with numpy : 0.00033855438232421875 nb_pixel_total : 5115 time to create 1 rle with old method : 0.006605625152587891 length of segment : 84 time for calcul the mask position with numpy : 0.0002124309539794922 nb_pixel_total : 2799 time to create 1 rle with old method : 0.0034084320068359375 length of segment : 86 time for calcul the mask position with numpy : 0.0012745857238769531 nb_pixel_total : 38378 time to create 1 rle with old method : 0.0438227653503418 length of segment : 217 time for calcul the mask position with numpy : 0.0003294944763183594 nb_pixel_total : 5757 time to create 1 rle with old method : 0.00675511360168457 length of segment : 121 time for calcul the mask position with numpy : 0.0019500255584716797 nb_pixel_total : 61229 time to create 1 rle with old method : 0.077606201171875 length of segment : 398 time for calcul the mask position with numpy : 0.00141143798828125 nb_pixel_total : 44100 time to create 1 rle with old method : 0.05189633369445801 length of segment : 200 time for calcul the mask position with numpy : 0.0003421306610107422 nb_pixel_total : 8049 time to create 1 rle with old method : 0.009601116180419922 length of segment : 79 time for calcul the mask position with numpy : 0.00016927719116210938 nb_pixel_total : 2827 time to create 1 rle with old method : 0.003567218780517578 length of segment : 42 time for calcul the mask position with numpy : 0.005536794662475586 nb_pixel_total : 170639 time to create 1 rle with new method : 0.008949518203735352 length of segment : 561 time for calcul the mask position with numpy : 0.0030562877655029297 nb_pixel_total : 139508 time to create 1 rle with old method : 0.17739510536193848 length of segment : 472 time for calcul the mask position with numpy : 0.004647493362426758 nb_pixel_total : 164634 time to create 1 rle with new method : 0.008719682693481445 length of segment : 567 time for calcul the mask position with numpy : 0.0001800060272216797 nb_pixel_total : 1362 time to create 1 rle with old method : 0.0024144649505615234 length of segment : 35 time for calcul the mask position with numpy : 0.00037360191345214844 nb_pixel_total : 6192 time to create 1 rle with old method : 0.010230302810668945 length of segment : 109 time for calcul the mask position with numpy : 0.001544952392578125 nb_pixel_total : 83362 time to create 1 rle with old method : 0.10159611701965332 length of segment : 305 time for calcul the mask position with numpy : 0.0002627372741699219 nb_pixel_total : 2141 time to create 1 rle with old method : 0.0027604103088378906 length of segment : 89 time for calcul the mask position with numpy : 0.0015540122985839844 nb_pixel_total : 54234 time to create 1 rle with old method : 0.061419010162353516 length of segment : 336 time for calcul the mask position with numpy : 0.0046999454498291016 nb_pixel_total : 170462 time to create 1 rle with new method : 0.007364511489868164 length of segment : 594 time for calcul the mask position with numpy : 0.0003063678741455078 nb_pixel_total : 4004 time to create 1 rle with old method : 0.00478363037109375 length of segment : 112 time for calcul the mask position with numpy : 0.0023908615112304688 nb_pixel_total : 106036 time to create 1 rle with old method : 0.11871170997619629 length of segment : 495 time for calcul the mask position with numpy : 0.0007033348083496094 nb_pixel_total : 24256 time to create 1 rle with old method : 0.027181148529052734 length of segment : 182 time for calcul the mask position with numpy : 0.0010600090026855469 nb_pixel_total : 58068 time to create 1 rle with old method : 0.06580448150634766 length of segment : 352 time spent for convertir_results : 18.959341764450073 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 167 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 51311 save missing photos in datou_result : time spend for datou_step_exec : 73.0286660194397 time spend to save output : 21.28157949447632 total time spend for step 1 : 94.31024551391602 step2:crop_condition Tue Mar 11 15:32:05 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 167 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 24 About to insert : list_path_to_insert length 24 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 34 About to insert : list_path_to_insert length 34 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1343022083, 1343022053, 1343021678, 1343021654, 1343021603, 1343021431, 1343021430, 1343021425, 1343021403, 1343021377] Looping around the photos to save general results len do output : 81 /-3705635498Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705635505Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705635507Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705635516Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705635501Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600609Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600616Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600608Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600610Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600645Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600630Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600638Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600633Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600644Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600641Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600631Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600632Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600680Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600682Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600677Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600694Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600704Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600705Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600711Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705635514Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600627Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705635522Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600619Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600662Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600708Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600714Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705635521Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600651Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600659Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600715Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705635509Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705635510Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705635517Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705635515Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705635512Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600613Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600612Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600628Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705635523Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600649Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600642Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600647Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600635Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600648Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600646Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600637Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600658Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600655Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600656Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600657Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600663Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600664Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705635526Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600668Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600667Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600685Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600687Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600684Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705635527Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600693Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600700Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600699Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600697Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600710Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705635511Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705635513Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705635504Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705635503Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600615Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600617Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600640Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600671Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600666Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600673Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600713Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705600643Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343022083', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343022053', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021678', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021654', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021603', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021431', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021430', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021425', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021403', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021377', None, None, None, None, None, '2658518') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 253 time used for this insertion : 0.04464364051818848 save_final save missing photos in datou_result : time spend for datou_step_exec : 55.57001328468323 time spend to save output : 0.04789233207702637 total time spend for step 2 : 55.617905616760254 step3:thcl Tue Mar 11 15:33:00 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 2 we are using the classfication for multi_thcl [2456, 2868] time to import caffe and check if the image exist : 0.006852149963378906 time to convert the images to numpy array : 4.0531158447265625e-06 time to import caffe and check if the image exist : 0.015252351760864258 time to convert the images to numpy array : 0.36261963844299316 time to import caffe and check if the image exist : 0.016509532928466797 time to convert the images to numpy array : 0.39021778106689453 time to import caffe and check if the image exist : 0.013807058334350586 time to convert the images to numpy array : 0.4058709144592285 time to import caffe and check if the image exist : 0.012369155883789062 time to convert the images to numpy array : 0.43427419662475586 time to import caffe and check if the image exist : 0.009810447692871094 time to convert the images to numpy array : 0.43766212463378906 time to import caffe and check if the image exist : 0.013435602188110352 time to convert the images to numpy array : 0.4360067844390869 time to import caffe and check if the image exist : 0.018282413482666016 time to convert the images to numpy array : 0.43611979484558105 time to import caffe and check if the image exist : 0.011742830276489258 time to convert the images to numpy array : 0.5763444900512695 time to import caffe and check if the image exist : 0.013150691986083984 time to convert the images to numpy array : 0.5819640159606934 total time to convert the images to numpy array : 0.9262423515319824 list photo_ids error: [] list photo_ids correct : [-3705635503, -3705600615, -3705600617, -3705600640, -3705600671, -3705600666, -3705600673, -3705600713, -3705600643, -3705635527, -3705600693, -3705600700, -3705600699, -3705600697, -3705600710, -3705635511, -3705635513, -3705635504, -3705600682, -3705600677, -3705600694, -3705600704, -3705600705, -3705600711, -3705635514, -3705600627, -3705635522, -3705635498, -3705635505, -3705635507, -3705635516, -3705635501, -3705600609, -3705600616, -3705600608, -3705600610, -3705600657, -3705600663, -3705600664, -3705635526, -3705600668, -3705600667, -3705600685, -3705600687, -3705600684, -3705635510, -3705635517, -3705635515, -3705635512, -3705600613, -3705600612, -3705600628, -3705635523, -3705600649, -3705600645, -3705600630, -3705600638, -3705600633, -3705600644, -3705600641, -3705600631, -3705600632, -3705600680, -3705600619, -3705600662, -3705600708, -3705600714, -3705635521, -3705600651, -3705600659, -3705600715, -3705635509, -3705600642, -3705600647, -3705600635, -3705600648, -3705600646, -3705600637, -3705600658, -3705600655, -3705600656] number of photos to traite : 81 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10372 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10153 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.360121250152588 time used to do the prediction : 0.23670196533203125 we don't save the descriptors for this thcl 2456 tagging for thcl : 2868 To do loadFromThcl(), then load ParamDescType : thcl2868 thcls : [{'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5288 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) To loadFromThcl() : net_5288 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10151 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_papier_nantes_300421 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_papier_nantes_300421 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_papier_nantes_300421 /data/models_weight/learn_papier_nantes_300421/caffemodel size_local : 44791983 size in s3 : 44791983 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:13 caffemodel already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/deploy.prototxt size_local : 17255 size in s3 : 17255 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:12 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:14 mean.npy already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/synset_words.txt size_local : 331 size in s3 : 331 create time local : 2021-08-09 05:56:00 create time in s3 : 2021-08-06 19:22:12 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_papier_nantes_300421/deploy.prototxt caffemodel_filename : /data/models_weight/learn_papier_nantes_300421/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10151 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.5236241817474365 time used to do the prediction : 0.1986980438232422 we don't save the descriptors for this thcl 2868 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 2.7894973754882812e-05 save missing photos in datou_result : time spend for datou_step_exec : 11.339838981628418 time spend to save output : 0.0006945133209228516 total time spend for step 3 : 11.34053349494934 step4:merge_mask_thcl_custom Tue Mar 11 15:33:11 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 167 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1343022083, 1343022053, 1343021678, 1343021654, 1343021603, 1343021431, 1343021430, 1343021425, 1343021403, 1343021377] Looping around the photos to save general results len do output : 10 /1343022083Didn't retrieve data .Didn't retrieve data . /1343022053Didn't retrieve data .Didn't retrieve data . /1343021678Didn't retrieve data .Didn't retrieve data . /1343021654Didn't retrieve data .Didn't retrieve data . /1343021603Didn't retrieve data .Didn't retrieve data . /1343021431Didn't retrieve data .Didn't retrieve data . /1343021430Didn't retrieve data .Didn't retrieve data . /1343021425Didn't retrieve data .Didn't retrieve data . /1343021403Didn't retrieve data .Didn't retrieve data . /1343021377Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343022083', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343022053', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021678', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021654', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021603', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021431', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021430', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021425', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021403', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021377', None, None, None, None, None, '2658518') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.01563429832458496 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.5306375026702881 time spend to save output : 0.016268253326416016 total time spend for step 4 : 0.5469057559967041 step5:rle_unique_nms_with_priority Tue Mar 11 15:33:12 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 4.1261749267578125 time for calcul the mask position with numpy : 0.3494558334350586 nb_pixel_total : 2863588 time to create 1 rle with new method : 0.6912260055541992 time for calcul the mask position with numpy : 0.0223538875579834 nb_pixel_total : 2047 time to create 1 rle with old method : 0.0031173229217529297 time for calcul the mask position with numpy : 0.022345781326293945 nb_pixel_total : 221190 time to create 1 rle with new method : 0.9945118427276611 time for calcul the mask position with numpy : 0.3354074954986572 nb_pixel_total : 3925441 time to create 1 rle with new method : 0.7874875068664551 time for calcul the mask position with numpy : 0.030910730361938477 nb_pixel_total : 37974 time to create 1 rle with old method : 0.039046287536621094 create new chi : 3.3538918495178223 time to delete rle : 0.014726877212524414 batch 1 Loaded 7 chid ids of type : 3726 Number RLEs to save : 14963 TO DO : save crop sub photo not yet done ! save time : 16.640713214874268 nb_obj : 2 nb_hashtags : 1 time to prepare the origin masks : 3.6158485412597656 time for calcul the mask position with numpy : 0.03292369842529297 nb_pixel_total : 1834108 time to create 1 rle with new method : 0.6417264938354492 time for calcul the mask position with numpy : 0.02175760269165039 nb_pixel_total : 14093 time to create 1 rle with old method : 0.0186159610748291 time for calcul the mask position with numpy : 0.49989891052246094 nb_pixel_total : 5202039 time to create 1 rle with new method : 0.4501993656158447 create new chi : 1.726210355758667 time to delete rle : 0.0010516643524169922 batch 1 Loaded 3 chid ids of type : 3726 Number RLEs to save : 12243 TO DO : save crop sub photo not yet done ! save time : 27.075568199157715 nb_obj : 25 nb_hashtags : 7 time to prepare the origin masks : 20.19362998008728 time for calcul the mask position with numpy : 0.3054220676422119 nb_pixel_total : 6241879 time to create 1 rle with new method : 0.6729388236999512 time for calcul the mask position with numpy : 0.041262149810791016 nb_pixel_total : 56282 time to create 1 rle with old method : 0.0668032169342041 time for calcul the mask position with numpy : 0.04549098014831543 nb_pixel_total : 721 time to create 1 rle with old method : 0.0010089874267578125 time for calcul the mask position with numpy : 0.04322338104248047 nb_pixel_total : 5895 time to create 1 rle with old method : 0.007021427154541016 time for calcul the mask position with numpy : 0.05009770393371582 nb_pixel_total : 234800 time to create 1 rle with new method : 0.8952949047088623 time for calcul the mask position with numpy : 0.037557125091552734 nb_pixel_total : 10200 time to create 1 rle with old method : 0.010417699813842773 time for calcul the mask position with numpy : 0.03763175010681152 nb_pixel_total : 6065 time to create 1 rle with old method : 0.006284236907958984 time for calcul the mask position with numpy : 0.03693795204162598 nb_pixel_total : 5437 time to create 1 rle with old method : 0.005628108978271484 time for calcul the mask position with numpy : 0.038125038146972656 nb_pixel_total : 1042 time to create 1 rle with old method : 0.0013513565063476562 time for calcul the mask position with numpy : 0.03745579719543457 nb_pixel_total : 26439 time to create 1 rle with old method : 0.027269840240478516 time for calcul the mask position with numpy : 0.03914642333984375 nb_pixel_total : 54 time to create 1 rle with old method : 0.00016355514526367188 time for calcul the mask position with numpy : 0.03871512413024902 nb_pixel_total : 42599 time to create 1 rle with old method : 0.04559135437011719 time for calcul the mask position with numpy : 0.0394132137298584 nb_pixel_total : 5923 time to create 1 rle with old method : 0.0064356327056884766 time for calcul the mask position with numpy : 0.04135560989379883 nb_pixel_total : 51981 time to create 1 rle with old method : 0.05494189262390137 time for calcul the mask position with numpy : 0.038516998291015625 nb_pixel_total : 24287 time to create 1 rle with old method : 0.02548384666442871 time for calcul the mask position with numpy : 0.03913593292236328 nb_pixel_total : 37902 time to create 1 rle with old method : 0.040094614028930664 time for calcul the mask position with numpy : 0.03877401351928711 nb_pixel_total : 74445 time to create 1 rle with old method : 0.07717609405517578 time for calcul the mask position with numpy : 0.04035043716430664 nb_pixel_total : 160 time to create 1 rle with old method : 0.0004019737243652344 time for calcul the mask position with numpy : 0.04039192199707031 nb_pixel_total : 60 time to create 1 rle with old method : 0.00015687942504882812 time for calcul the mask position with numpy : 0.03725743293762207 nb_pixel_total : 13325 time to create 1 rle with old method : 0.014125347137451172 time for calcul the mask position with numpy : 0.04355025291442871 nb_pixel_total : 3800 time to create 1 rle with old method : 0.004091739654541016 time for calcul the mask position with numpy : 0.04091954231262207 nb_pixel_total : 45471 time to create 1 rle with old method : 0.04742240905761719 time for calcul the mask position with numpy : 0.023418188095092773 nb_pixel_total : 2110 time to create 1 rle with old method : 0.0024209022521972656 time for calcul the mask position with numpy : 0.024489402770996094 nb_pixel_total : 386 time to create 1 rle with old method : 0.0005857944488525391 time for calcul the mask position with numpy : 0.023998737335205078 nb_pixel_total : 4553 time to create 1 rle with old method : 0.0049381256103515625 time for calcul the mask position with numpy : 0.02594614028930664 nb_pixel_total : 154424 time to create 1 rle with new method : 0.7918939590454102 create new chi : 4.156177282333374 time to delete rle : 0.0020742416381835938 batch 1 Loaded 46 chid ids of type : 3726 Number RLEs to save : 14009 TO DO : save crop sub photo not yet done ! save time : 17.73496389389038 nb_obj : 20 nb_hashtags : 5 time to prepare the origin masks : 15.339566469192505 time for calcul the mask position with numpy : 0.834749698638916 nb_pixel_total : 6297489 time to create 1 rle with new method : 1.015733003616333 time for calcul the mask position with numpy : 0.02444624900817871 nb_pixel_total : 25784 time to create 1 rle with old method : 0.03245115280151367 time for calcul the mask position with numpy : 0.0257260799407959 nb_pixel_total : 10453 time to create 1 rle with old method : 0.012480020523071289 time for calcul the mask position with numpy : 0.025787830352783203 nb_pixel_total : 1613 time to create 1 rle with old method : 0.0020308494567871094 time for calcul the mask position with numpy : 0.02270650863647461 nb_pixel_total : 886 time to create 1 rle with old method : 0.0009765625 time for calcul the mask position with numpy : 0.023838043212890625 nb_pixel_total : 4631 time to create 1 rle with old method : 0.005443096160888672 time for calcul the mask position with numpy : 0.023366451263427734 nb_pixel_total : 19221 time to create 1 rle with old method : 0.0204923152923584 time for calcul the mask position with numpy : 0.025422334671020508 nb_pixel_total : 14743 time to create 1 rle with old method : 0.016017913818359375 time for calcul the mask position with numpy : 0.023628950119018555 nb_pixel_total : 3228 time to create 1 rle with old method : 0.0035147666931152344 time for calcul the mask position with numpy : 0.02447342872619629 nb_pixel_total : 214986 time to create 1 rle with new method : 0.6360840797424316 time for calcul the mask position with numpy : 0.02301621437072754 nb_pixel_total : 47541 time to create 1 rle with old method : 0.05081653594970703 time for calcul the mask position with numpy : 0.0228426456451416 nb_pixel_total : 52091 time to create 1 rle with old method : 0.05488872528076172 time for calcul the mask position with numpy : 0.02367234230041504 nb_pixel_total : 1933 time to create 1 rle with old method : 0.002302885055541992 time for calcul the mask position with numpy : 0.028380155563354492 nb_pixel_total : 229 time to create 1 rle with old method : 0.00033020973205566406 time for calcul the mask position with numpy : 0.02397894859313965 nb_pixel_total : 23642 time to create 1 rle with old method : 0.025041818618774414 time for calcul the mask position with numpy : 0.024254798889160156 nb_pixel_total : 13091 time to create 1 rle with old method : 0.014057159423828125 time for calcul the mask position with numpy : 0.03494429588317871 nb_pixel_total : 5779 time to create 1 rle with old method : 0.0062334537506103516 time for calcul the mask position with numpy : 0.037893056869506836 nb_pixel_total : 28734 time to create 1 rle with old method : 0.030882835388183594 time for calcul the mask position with numpy : 0.045886993408203125 nb_pixel_total : 136765 time to create 1 rle with old method : 0.14905548095703125 time for calcul the mask position with numpy : 0.04153323173522949 nb_pixel_total : 55794 time to create 1 rle with old method : 0.06555294990539551 time for calcul the mask position with numpy : 0.03997945785522461 nb_pixel_total : 91607 time to create 1 rle with old method : 0.09673595428466797 create new chi : 3.711855411529541 time to delete rle : 0.0013518333435058594 Catched exception ! Connect or reconnect ! batch 1 Loaded 26 chid ids of type : 3726 Number RLEs to save : 12751 TO DO : save crop sub photo not yet done ! save time : 13.5696861743927 nb_obj : 22 nb_hashtags : 6 time to prepare the origin masks : 18.650365114212036 time for calcul the mask position with numpy : 0.21828389167785645 nb_pixel_total : 5794724 time to create 1 rle with new method : 0.2982823848724365 time for calcul the mask position with numpy : 0.03872799873352051 nb_pixel_total : 10283 time to create 1 rle with old method : 0.011481046676635742 time for calcul the mask position with numpy : 0.0350492000579834 nb_pixel_total : 10891 time to create 1 rle with old method : 0.012142658233642578 time for calcul the mask position with numpy : 0.0387265682220459 nb_pixel_total : 38151 time to create 1 rle with old method : 0.043065786361694336 time for calcul the mask position with numpy : 0.03590130805969238 nb_pixel_total : 99752 time to create 1 rle with old method : 0.11521124839782715 time for calcul the mask position with numpy : 0.022013187408447266 nb_pixel_total : 1381 time to create 1 rle with old method : 0.0017054080963134766 time for calcul the mask position with numpy : 0.021625757217407227 nb_pixel_total : 29055 time to create 1 rle with old method : 0.03289031982421875 time for calcul the mask position with numpy : 0.022177696228027344 nb_pixel_total : 16695 time to create 1 rle with old method : 0.018813133239746094 time for calcul the mask position with numpy : 0.02345561981201172 nb_pixel_total : 41440 time to create 1 rle with old method : 0.05346393585205078 time for calcul the mask position with numpy : 0.02322244644165039 nb_pixel_total : 18094 time to create 1 rle with old method : 0.019886493682861328 time for calcul the mask position with numpy : 0.024873733520507812 nb_pixel_total : 173289 time to create 1 rle with new method : 1.5068964958190918 time for calcul the mask position with numpy : 0.04577898979187012 nb_pixel_total : 5785 time to create 1 rle with old method : 0.018686294555664062 time for calcul the mask position with numpy : 0.049178123474121094 nb_pixel_total : 3257 time to create 1 rle with old method : 0.0036840438842773438 time for calcul the mask position with numpy : 0.05849814414978027 nb_pixel_total : 91150 time to create 1 rle with old method : 0.1639854907989502 time for calcul the mask position with numpy : 0.044365644454956055 nb_pixel_total : 54721 time to create 1 rle with old method : 0.09966802597045898 time for calcul the mask position with numpy : 0.037317752838134766 nb_pixel_total : 53944 time to create 1 rle with old method : 0.07146859169006348 time for calcul the mask position with numpy : 0.039629220962524414 nb_pixel_total : 141330 time to create 1 rle with old method : 0.18748092651367188 time for calcul the mask position with numpy : 0.046027183532714844 nb_pixel_total : 13854 time to create 1 rle with old method : 0.016313791275024414 time for calcul the mask position with numpy : 0.039305925369262695 nb_pixel_total : 110064 time to create 1 rle with old method : 0.12569069862365723 time for calcul the mask position with numpy : 0.03922319412231445 nb_pixel_total : 109 time to create 1 rle with old method : 0.00023031234741210938 time for calcul the mask position with numpy : 0.040393829345703125 nb_pixel_total : 3678 time to create 1 rle with old method : 0.004111528396606445 time for calcul the mask position with numpy : 0.04143714904785156 nb_pixel_total : 178112 time to create 1 rle with new method : 0.5935614109039307 time for calcul the mask position with numpy : 0.04680013656616211 nb_pixel_total : 160481 time to create 1 rle with new method : 0.5267999172210693 create new chi : 5.0837531089782715 time to delete rle : 0.0035767555236816406 batch 1 Loaded 23 chid ids of type : 3726 Number RLEs to save : 18887 TO DO : save crop sub photo not yet done ! save time : 7.367767572402954 nb_obj : 7 nb_hashtags : 4 time to prepare the origin masks : 6.455276250839233 time for calcul the mask position with numpy : 0.28609323501586914 nb_pixel_total : 6048501 time to create 1 rle with new method : 0.4071948528289795 time for calcul the mask position with numpy : 0.054792165756225586 nb_pixel_total : 696461 time to create 1 rle with new method : 0.5313992500305176 time for calcul the mask position with numpy : 0.03629565238952637 nb_pixel_total : 40991 time to create 1 rle with old method : 0.04494905471801758 time for calcul the mask position with numpy : 0.03551340103149414 nb_pixel_total : 32729 time to create 1 rle with old method : 0.03620433807373047 time for calcul the mask position with numpy : 0.03609752655029297 nb_pixel_total : 130180 time to create 1 rle with old method : 0.14708709716796875 time for calcul the mask position with numpy : 0.02190399169921875 nb_pixel_total : 2315 time to create 1 rle with old method : 0.0027489662170410156 time for calcul the mask position with numpy : 0.02399444580078125 nb_pixel_total : 48457 time to create 1 rle with old method : 0.07765936851501465 time for calcul the mask position with numpy : 0.02275228500366211 nb_pixel_total : 50606 time to create 1 rle with old method : 0.05981016159057617 create new chi : 1.887387990951538 time to delete rle : 0.0009012222290039062 batch 1 Loaded 8 chid ids of type : 3726 Number RLEs to save : 8195 TO DO : save crop sub photo not yet done ! save time : 11.442028045654297 nb_obj : 14 nb_hashtags : 7 time to prepare the origin masks : 7.5568413734436035 time for calcul the mask position with numpy : 0.3522977828979492 nb_pixel_total : 6307505 time to create 1 rle with new method : 0.5543696880340576 time for calcul the mask position with numpy : 0.029456138610839844 nb_pixel_total : 17574 time to create 1 rle with old method : 0.019588708877563477 time for calcul the mask position with numpy : 0.04108881950378418 nb_pixel_total : 141930 time to create 1 rle with old method : 0.1999962329864502 time for calcul the mask position with numpy : 0.032683372497558594 nb_pixel_total : 119679 time to create 1 rle with old method : 0.1447601318359375 time for calcul the mask position with numpy : 0.03132128715515137 nb_pixel_total : 626 time to create 1 rle with old method : 0.0010657310485839844 time for calcul the mask position with numpy : 0.027463197708129883 nb_pixel_total : 45130 time to create 1 rle with old method : 0.05107235908508301 time for calcul the mask position with numpy : 0.026971101760864258 nb_pixel_total : 8394 time to create 1 rle with old method : 0.009994745254516602 time for calcul the mask position with numpy : 0.02609729766845703 nb_pixel_total : 6590 time to create 1 rle with old method : 0.007942676544189453 time for calcul the mask position with numpy : 0.030483245849609375 nb_pixel_total : 41905 time to create 1 rle with old method : 0.05437898635864258 time for calcul the mask position with numpy : 0.02693462371826172 nb_pixel_total : 2636 time to create 1 rle with old method : 0.0034427642822265625 time for calcul the mask position with numpy : 0.026144027709960938 nb_pixel_total : 17609 time to create 1 rle with old method : 0.019859790802001953 time for calcul the mask position with numpy : 0.0270078182220459 nb_pixel_total : 53886 time to create 1 rle with old method : 0.059197187423706055 time for calcul the mask position with numpy : 0.02298116683959961 nb_pixel_total : 15824 time to create 1 rle with old method : 0.016779422760009766 time for calcul the mask position with numpy : 0.025023937225341797 nb_pixel_total : 254916 time to create 1 rle with new method : 0.4179575443267822 time for calcul the mask position with numpy : 0.03293251991271973 nb_pixel_total : 16036 time to create 1 rle with old method : 0.025364160537719727 create new chi : 2.4284846782684326 time to delete rle : 0.002293109893798828 batch 1 Loaded 16 chid ids of type : 3726 Number RLEs to save : 9545 TO DO : save crop sub photo not yet done ! save time : 13.598357677459717 nb_obj : 12 nb_hashtags : 4 time to prepare the origin masks : 9.987964630126953 time for calcul the mask position with numpy : 0.8033690452575684 nb_pixel_total : 6689107 time to create 1 rle with new method : 1.0705358982086182 time for calcul the mask position with numpy : 0.03630328178405762 nb_pixel_total : 12554 time to create 1 rle with old method : 0.014444828033447266 time for calcul the mask position with numpy : 0.04128861427307129 nb_pixel_total : 104097 time to create 1 rle with old method : 0.11259889602661133 time for calcul the mask position with numpy : 0.0399622917175293 nb_pixel_total : 14935 time to create 1 rle with old method : 0.016030550003051758 time for calcul the mask position with numpy : 0.03872060775756836 nb_pixel_total : 1094 time to create 1 rle with old method : 0.0011837482452392578 time for calcul the mask position with numpy : 0.03776240348815918 nb_pixel_total : 80685 time to create 1 rle with old method : 0.08371329307556152 time for calcul the mask position with numpy : 0.035315752029418945 nb_pixel_total : 1388 time to create 1 rle with old method : 0.0016117095947265625 time for calcul the mask position with numpy : 0.02561044692993164 nb_pixel_total : 13589 time to create 1 rle with old method : 0.015273094177246094 time for calcul the mask position with numpy : 0.0225069522857666 nb_pixel_total : 20193 time to create 1 rle with old method : 0.021206140518188477 time for calcul the mask position with numpy : 0.02231287956237793 nb_pixel_total : 2301 time to create 1 rle with old method : 0.0024340152740478516 time for calcul the mask position with numpy : 0.022774219512939453 nb_pixel_total : 29619 time to create 1 rle with old method : 0.03199410438537598 time for calcul the mask position with numpy : 0.023281097412109375 nb_pixel_total : 6210 time to create 1 rle with old method : 0.006813764572143555 time for calcul the mask position with numpy : 0.024655580520629883 nb_pixel_total : 74468 time to create 1 rle with old method : 0.0778207778930664 create new chi : 2.6696226596832275 time to delete rle : 0.0008199214935302734 batch 1 Loaded 13 chid ids of type : 3726 Number RLEs to save : 6687 TO DO : save crop sub photo not yet done ! save time : 5.252331972122192 nb_obj : 14 nb_hashtags : 5 time to prepare the origin masks : 9.000022649765015 time for calcul the mask position with numpy : 0.965446949005127 nb_pixel_total : 6532861 time to create 1 rle with new method : 0.8536515235900879 time for calcul the mask position with numpy : 0.044774770736694336 nb_pixel_total : 272097 time to create 1 rle with new method : 0.5842525959014893 time for calcul the mask position with numpy : 0.03472542762756348 nb_pixel_total : 44002 time to create 1 rle with old method : 0.051136016845703125 time for calcul the mask position with numpy : 0.03322243690490723 nb_pixel_total : 2805 time to create 1 rle with old method : 0.0031943321228027344 time for calcul the mask position with numpy : 0.033139944076538086 nb_pixel_total : 8024 time to create 1 rle with old method : 0.009564399719238281 time for calcul the mask position with numpy : 0.035979509353637695 nb_pixel_total : 5741 time to create 1 rle with old method : 0.006626129150390625 time for calcul the mask position with numpy : 0.033982276916503906 nb_pixel_total : 7139 time to create 1 rle with old method : 0.008318185806274414 time for calcul the mask position with numpy : 0.02971363067626953 nb_pixel_total : 2785 time to create 1 rle with old method : 0.0031342506408691406 time for calcul the mask position with numpy : 0.03759884834289551 nb_pixel_total : 5055 time to create 1 rle with old method : 0.005664348602294922 time for calcul the mask position with numpy : 0.03141617774963379 nb_pixel_total : 61222 time to create 1 rle with old method : 0.06725835800170898 time for calcul the mask position with numpy : 0.025124073028564453 nb_pixel_total : 38221 time to create 1 rle with old method : 0.04467439651489258 time for calcul the mask position with numpy : 0.033669471740722656 nb_pixel_total : 3100 time to create 1 rle with old method : 0.003625154495239258 time for calcul the mask position with numpy : 0.02227044105529785 nb_pixel_total : 9977 time to create 1 rle with old method : 0.011719226837158203 time for calcul the mask position with numpy : 0.022868871688842773 nb_pixel_total : 11172 time to create 1 rle with old method : 0.013092517852783203 time for calcul the mask position with numpy : 0.023307085037231445 nb_pixel_total : 46039 time to create 1 rle with old method : 0.053822994232177734 create new chi : 3.204927921295166 time to delete rle : 0.0009932518005371094 batch 1 Loaded 16 chid ids of type : 3726 Number RLEs to save : 7712 TO DO : save crop sub photo not yet done ! save time : 10.237706184387207 nb_obj : 12 nb_hashtags : 7 time to prepare the origin masks : 8.099809885025024 time for calcul the mask position with numpy : 0.6725332736968994 nb_pixel_total : 6291482 time to create 1 rle with new method : 0.5357203483581543 time for calcul the mask position with numpy : 0.023355960845947266 nb_pixel_total : 105908 time to create 1 rle with old method : 0.1166391372680664 time for calcul the mask position with numpy : 0.02277684211730957 nb_pixel_total : 515 time to create 1 rle with old method : 0.0008654594421386719 time for calcul the mask position with numpy : 0.022724390029907227 nb_pixel_total : 170061 time to create 1 rle with new method : 0.4032175540924072 time for calcul the mask position with numpy : 0.023450136184692383 nb_pixel_total : 24204 time to create 1 rle with old method : 0.027091503143310547 time for calcul the mask position with numpy : 0.0225679874420166 nb_pixel_total : 82890 time to create 1 rle with old method : 0.0850222110748291 time for calcul the mask position with numpy : 0.02852940559387207 nb_pixel_total : 57937 time to create 1 rle with old method : 0.06876540184020996 time for calcul the mask position with numpy : 0.027665138244628906 nb_pixel_total : 3996 time to create 1 rle with old method : 0.026523113250732422 time for calcul the mask position with numpy : 0.03274369239807129 nb_pixel_total : 2128 time to create 1 rle with old method : 0.002948284149169922 time for calcul the mask position with numpy : 0.030710697174072266 nb_pixel_total : 6171 time to create 1 rle with old method : 0.008479118347167969 time for calcul the mask position with numpy : 0.03043985366821289 nb_pixel_total : 1352 time to create 1 rle with old method : 0.0018973350524902344 time for calcul the mask position with numpy : 0.03211665153503418 nb_pixel_total : 139328 time to create 1 rle with old method : 0.1534111499786377 time for calcul the mask position with numpy : 0.023961544036865234 nb_pixel_total : 164268 time to create 1 rle with new method : 0.786151647567749 create new chi : 3.309300661087036 time to delete rle : 0.0011048316955566406 batch 1 Loaded 13 chid ids of type : 3726 Number RLEs to save : 8984 TO DO : save crop sub photo not yet done ! save time : 11.490326166152954 map_output_result : {1343022083: (0.0, 'Should be the crop_list due to order', 0), 1343022053: (0.0, 'Should be the crop_list due to order', 0), 1343021678: (0.0, 'Should be the crop_list due to order', 0), 1343021654: (0.0, 'Should be the crop_list due to order', 0), 1343021603: (0.0, 'Should be the crop_list due to order', 0), 1343021431: (0.0, 'Should be the crop_list due to order', 0), 1343021430: (0.0, 'Should be the crop_list due to order', 0), 1343021425: (0.0, 'Should be the crop_list due to order', 0), 1343021403: (0.0, 'Should be the crop_list due to order', 0), 1343021377: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1343022083, 1343022053, 1343021678, 1343021654, 1343021603, 1343021431, 1343021430, 1343021425, 1343021403, 1343021377] Looping around the photos to save general results len do output : 10 /1343022083.Didn't retrieve data . /1343022053.Didn't retrieve data . /1343021678.Didn't retrieve data . /1343021654.Didn't retrieve data . /1343021603.Didn't retrieve data . /1343021431.Didn't retrieve data . /1343021430.Didn't retrieve data . /1343021425.Didn't retrieve data . /1343021403.Didn't retrieve data . /1343021377.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343022083', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343022053', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021678', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021654', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021603', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021431', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021430', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021425', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021403', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021377', None, None, None, None, None, '2658518') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.014208078384399414 save_final save missing photos in datou_result : time spend for datou_step_exec : 270.27824425697327 time spend to save output : 0.015979528427124023 total time spend for step 5 : 270.2942237854004 step6:crop_condition Tue Mar 11 15:37:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 171 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 15 About to insert : list_path_to_insert length 15 new photo from crops ! About to upload 15 photos upload in portfolio : 4869462 init cache_photo without model_param we have 15 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741703875_1438157 we have uploaded 15 photos in the portfolio 4869462 time of upload the photos Elapsed time : 5.200321912765503 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 10 About to insert : list_path_to_insert length 10 new photo from crops ! About to upload 10 photos upload in portfolio : 4869462 init cache_photo without model_param we have 10 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741703884_1438157 we have uploaded 10 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.376574993133545 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741703890_1438157 we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.3684825897216797 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741703895_1438157 we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.3179032802581787 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 10 About to insert : list_path_to_insert length 10 new photo from crops ! About to upload 10 photos upload in portfolio : 4869462 init cache_photo without model_param we have 10 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741703899_1438157 we have uploaded 10 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.0123038291931152 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741703903_1438157 we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.5481011867523193 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741703905_1438157 we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.9762759208679199 we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1343022083, 1343022053, 1343021678, 1343021654, 1343021603, 1343021431, 1343021430, 1343021425, 1343021403, 1343021377] Looping around the photos to save general results len do output : 46 /1343073871Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073872Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073874Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073875Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073876Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073878Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073879Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073880Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073882Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073883Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073884Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073886Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073887Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073888Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073890Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073942Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073943Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073945Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073946Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073947Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073949Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073950Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073952Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073953Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073954Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073964Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073965Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073966Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073967Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073970Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073971Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073972Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073973Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073982Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073983Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073985Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073986Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073987Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073989Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073990Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073991Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073993Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343073994Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343074007Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343074016Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343074018Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343022083', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343022053', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021678', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021654', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021603', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021431', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021430', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021425', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021403', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021377', None, None, None, None, None, '2658518') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 148 time used for this insertion : 0.0324404239654541 save_final save missing photos in datou_result : time spend for datou_step_exec : 51.36023306846619 time spend to save output : 0.03750276565551758 total time spend for step 6 : 51.397735834121704 step7:ventilate_hashtags_in_portfolio Tue Mar 11 15:38:34 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 21291428 get user id for portfolio 21291428 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21291428 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('cartonnette','Carton_gris','Teint_Dans_La_Masse','Carton_brun','mal_croppe','environnement','kraft','papier','metal','plastique','flou','autre_refus')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21291428 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('cartonnette','Carton_gris','Teint_Dans_La_Masse','Carton_brun','mal_croppe','environnement','kraft','papier','metal','plastique','flou','autre_refus')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/21292097,21292098,21292099,21292100,21292101,21292103,21292104,21292105,21292106,21292108,21292109,21292110?tags=cartonnette,papier,autre_refus,flou,Carton_brun,Teint_Dans_La_Masse,kraft,plastique,environnement,Carton_gris,mal_croppe,metal Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1343022083, 1343022053, 1343021678, 1343021654, 1343021603, 1343021431, 1343021430, 1343021425, 1343021403, 1343021377] Looping around the photos to save general results len do output : 1 /21291428. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343022083', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343022053', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021678', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021654', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021603', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021431', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021430', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021425', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021403', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021377', None, None, None, None, None, '2658518') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.1336536407470703 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.524301290512085 time spend to save output : 0.13408470153808594 total time spend for step 7 : 1.658385992050171 step8:final Tue Mar 11 15:38:35 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1343022083: ('0.0801603994846114',), 1343022053: ('0.0801603994846114',), 1343021678: ('0.0801603994846114',), 1343021654: ('0.0801603994846114',), 1343021603: ('0.0801603994846114',), 1343021431: ('0.0801603994846114',), 1343021430: ('0.0801603994846114',), 1343021425: ('0.0801603994846114',), 1343021403: ('0.0801603994846114',), 1343021377: ('0.0801603994846114',)} new output for save of step final : {1343022083: ('0.0801603994846114',), 1343022053: ('0.0801603994846114',), 1343021678: ('0.0801603994846114',), 1343021654: ('0.0801603994846114',), 1343021603: ('0.0801603994846114',), 1343021431: ('0.0801603994846114',), 1343021430: ('0.0801603994846114',), 1343021425: ('0.0801603994846114',), 1343021403: ('0.0801603994846114',), 1343021377: ('0.0801603994846114',)} [1343022083, 1343022053, 1343021678, 1343021654, 1343021603, 1343021431, 1343021430, 1343021425, 1343021403, 1343021377] Looping around the photos to save general results len do output : 10 /1343022083.Didn't retrieve data . /1343022053.Didn't retrieve data . /1343021678.Didn't retrieve data . /1343021654.Didn't retrieve data . /1343021603.Didn't retrieve data . /1343021431.Didn't retrieve data . /1343021430.Didn't retrieve data . /1343021425.Didn't retrieve data . /1343021403.Didn't retrieve data . /1343021377.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343022083', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343022053', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021678', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021654', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021603', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021431', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021430', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021425', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021403', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021377', None, None, None, None, None, '2658518') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.014530181884765625 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.11800980567932129 time spend to save output : 0.01503896713256836 total time spend for step 8 : 0.13304877281188965 step9:velours_tree Tue Mar 11 15:38:36 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.13329124450683594 time spend to save output : 4.601478576660156e-05 total time spend for step 9 : 0.13333725929260254 step10:send_mail_cod Tue Mar 11 15:38:36 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P21291428_11-03-2025_15_38_36.pdf 21292097 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212920971741703916 21292099 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212920991741703916 21292100 imagette212921001741703918 21292104 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212921041741703918 21292105 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212921051741703918 21292109 imagette212921091741703918 21292110 imagette212921101741703918 21292101 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212921011741703918 21292103 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212921031741703919 21292108 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212921081741703921 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=21291428 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/21292097,21292098,21292099,21292100,21292101,21292103,21292104,21292105,21292106,21292108,21292109,21292110?tags=cartonnette,papier,autre_refus,flou,Carton_brun,Teint_Dans_La_Masse,kraft,plastique,environnement,Carton_gris,mal_croppe,metal your option no_mail is active, we will not send the real mail to your client args[1343022083] : ('0.0801603994846114',) no score found for photo 1343022083 We are sending mail with results at report@fotonower.com args[1343022053] : ('0.0801603994846114',) no score found for photo 1343022053 We are sending mail with results at report@fotonower.com args[1343021678] : ('0.0801603994846114',) no score found for photo 1343021678 We are sending mail with results at report@fotonower.com args[1343021654] : ('0.0801603994846114',) no score found for photo 1343021654 We are sending mail with results at report@fotonower.com args[1343021603] : ('0.0801603994846114',) no score found for photo 1343021603 We are sending mail with results at report@fotonower.com args[1343021431] : ('0.0801603994846114',) no score found for photo 1343021431 We are sending mail with results at report@fotonower.com args[1343021430] : ('0.0801603994846114',) no score found for photo 1343021430 We are sending mail with results at report@fotonower.com args[1343021425] : ('0.0801603994846114',) no score found for photo 1343021425 We are sending mail with results at report@fotonower.com args[1343021403] : ('0.0801603994846114',) no score found for photo 1343021403 We are sending mail with results at report@fotonower.com args[1343021377] : ('0.0801603994846114',) no score found for photo 1343021377 We are sending mail with results at report@fotonower.com refus_total : 0.0801603994846114 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21291428 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1343021425,1343021430,1343021654,1343021678,1343022749,1343022747,1343021377,1343022217,1343022531,1343022739,1343023075,1343022775,1343022175,1343022101,1343023136,1343022083,1343022053,1343021603,1343021431,1343021403) Found this number of photos: 20 begin to download photo : 1343021425 begin to download photo : 1343022747 begin to download photo : 1343023075 begin to download photo : 1343022083 download finish for photo 1343021425 begin to download photo : 1343021430 download finish for photo 1343022747 begin to download photo : 1343021377 download finish for photo 1343023075 begin to download photo : 1343022775 download finish for photo 1343021430 begin to download photo : 1343021654 download finish for photo 1343022083 begin to download photo : 1343022053 download finish for photo 1343021377 begin to download photo : 1343022217 download finish for photo 1343022775 begin to download photo : 1343022175 download finish for photo 1343022053 begin to download photo : 1343021603 download finish for photo 1343021654 begin to download photo : 1343021678 download finish for photo 1343022217 begin to download photo : 1343022531 download finish for photo 1343021603 begin to download photo : 1343021431 download finish for photo 1343021678 begin to download photo : 1343022749 download finish for photo 1343022175 begin to download photo : 1343022101 download finish for photo 1343022531 begin to download photo : 1343022739 download finish for photo 1343021431 begin to download photo : 1343021403 download finish for photo 1343022101 begin to download photo : 1343023136 download finish for photo 1343022739 download finish for photo 1343022749 download finish for photo 1343021403 download finish for photo 1343023136 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21291428_11-03-2025_15_38_36.pdf results_Auto_P21291428_11-03-2025_15_38_36.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21291428_11-03-2025_15_38_36.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3459','21291428','results_Auto_P21291428_11-03-2025_15_38_36.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21291428_11-03-2025_15_38_36.pdf','pdf','','1.03','0.0801603994846114') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1343022083, 1343022053, 1343021678, 1343021654, 1343021603, 1343021431, 1343021430, 1343021425, 1343021403, 1343021377] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343022083', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343022053', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021678', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021654', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021603', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021431', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021430', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021425', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021403', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021377', None, None, None, None, None, '2658518') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.013720035552978516 save_final save missing photos in datou_result : time spend for datou_step_exec : 11.262304306030273 time spend to save output : 0.013952493667602539 total time spend for step 10 : 11.276256799697876 step11:split_time_score Tue Mar 11 15:38:47 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('13', 20),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 11032025 21291428 Nombre de photos uploadées : 20 / 23040 (0%) 11032025 21291428 Nombre de photos taguées (types de déchets): 0 / 20 (0%) 11032025 21291428 Nombre de photos taguées (volume) : 0 / 20 (0%) elapsed_time : load_data_split_time_score 1.6689300537109375e-06 elapsed_time : order_list_meta_photo_and_scores 5.9604644775390625e-06 ???????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0012135505676269531 elapsed_time : insert_dashboard_record_day_entry 0.028424501419067383 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.05881318002080126 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21291426_11-03-2025_15_35_46.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21291426 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21291426 AND mptpi.`type`=4207 To do Qualite : 0.07848604537653218 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21268995_11-03-2025_10_04_47.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21268995 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21268995 AND mptpi.`type`=4207 To do Qualite : 0.21508235889842053 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21268996_11-03-2025_09_21_12.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21268996 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21268996 AND mptpi.`type`=3594 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21268997_11-03-2025_09_36_04.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21268997 order by id desc limit 1 Qualite : 0.2996323014615577 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21268998_11-03-2025_09_21_30.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21268998 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21268998 AND mptpi.`type`=4200 To do Qualite : 0.059107923268759154 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21269000_11-03-2025_09_44_59.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21269000 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11488 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11496 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11497 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11495 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11495 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11489 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11489 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11575 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11575 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11491 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11490 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11490 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11498 send_mail_cod have less outputs used (0) than in the step definition (1) : some outputs may be not used ! Step 11499 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11491 doesn't seem to be define in the database( WARNING : type of input 3 of step 11490 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11488 doesn't seem to be define in the database( WARNING : type of input 2 of step 11492 doesn't seem to be define in the database( WARNING : output 1 of step 11488 have datatype=2 whereas input 1 of step 11495 have datatype=7 WARNING : type of output 2 of step 11495 doesn't seem to be define in the database( WARNING : type of input 1 of step 11489 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11491 have datatype=10 whereas input 3 of step 11498 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11575 doesn't seem to be define in the database( WARNING : output 1 of step 11489 have datatype=7 whereas input 2 of step 11575 have datatype=None WARNING : type of output 3 of step 11575 doesn't seem to be define in the database( WARNING : type of input 1 of step 11491 doesn't seem to be define in the database( WARNING : output 0 of step 11491 have datatype=10 whereas input 0 of step 11581 have datatype=18 WARNING : type of input 5 of step 11498 doesn't seem to be define in the database( WARNING : output 0 of step 11581 have datatype=11 whereas input 5 of step 11498 have datatype=None WARNING : type of output 1 of step 11496 doesn't seem to be define in the database( WARNING : type of input 3 of step 11492 doesn't seem to be define in the database( WARNING : type of output 1 of step 11497 doesn't seem to be define in the database( WARNING : type of input 3 of step 11492 doesn't seem to be define in the database( WARNING : output 0 of step 11495 have datatype=1 whereas input 0 of step 11489 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21269000 AND mptpi.`type`=4209 To do Qualite : 0.09529821171173529 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21269001_11-03-2025_09_17_20.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21269001 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21269001 AND mptpi.`type`=3726 To do Qualite : 0.12370809963529884 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21270945_11-03-2025_12_54_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21270945 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21270945 AND mptpi.`type`=3327 To do Qualite : 0.044100679011939585 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21276914_11-03-2025_11_38_36.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21276914 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21276914 AND mptpi.`type`=4207 To do Qualite : 0.04798165055335807 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21276915_11-03-2025_12_41_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21276915 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21276915 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290022 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290047 order by id desc limit 1 Qualite : 0.24539234952758107 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282296_11-03-2025_13_26_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282296 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21282296 AND mptpi.`type`=3594 To do Qualite : 0.23651912322606505 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282297_11-03-2025_14_28_01.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282297 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21282297 AND mptpi.`type`=3594 To do Qualite : 0.30571381203013953 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282298_11-03-2025_13_24_48.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282298 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21282298 AND mptpi.`type`=4200 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282299_11-03-2025_12_43_10.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282299 order by id desc limit 1 Qualite : 0.3846979317267506 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282300_11-03-2025_13_27_08.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282300 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21282300 AND mptpi.`type`=4200 To do Qualite : 0.3844764824714978 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282301_11-03-2025_13_46_16.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282301 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21282301 AND mptpi.`type`=4200 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282303_11-03-2025_12_44_04.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282303 order by id desc limit 1 Qualite : 0.21547889555127603 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21290048_11-03-2025_15_09_00.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290048 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21290048 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290049 order by id desc limit 1 Qualite : 0.056030119320647845 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21290050_11-03-2025_15_30_25.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290050 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11488 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11496 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11497 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11495 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11495 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11489 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11489 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11575 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11575 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11491 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11490 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11490 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11498 send_mail_cod have less outputs used (0) than in the step definition (1) : some outputs may be not used ! Step 11499 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11491 doesn't seem to be define in the database( WARNING : type of input 3 of step 11490 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11488 doesn't seem to be define in the database( WARNING : type of input 2 of step 11492 doesn't seem to be define in the database( WARNING : output 1 of step 11488 have datatype=2 whereas input 1 of step 11495 have datatype=7 WARNING : type of output 2 of step 11495 doesn't seem to be define in the database( WARNING : type of input 1 of step 11489 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11491 have datatype=10 whereas input 3 of step 11498 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11575 doesn't seem to be define in the database( WARNING : output 1 of step 11489 have datatype=7 whereas input 2 of step 11575 have datatype=None WARNING : type of output 3 of step 11575 doesn't seem to be define in the database( WARNING : type of input 1 of step 11491 doesn't seem to be define in the database( WARNING : output 0 of step 11491 have datatype=10 whereas input 0 of step 11581 have datatype=18 WARNING : type of input 5 of step 11498 doesn't seem to be define in the database( WARNING : output 0 of step 11581 have datatype=11 whereas input 5 of step 11498 have datatype=None WARNING : type of output 1 of step 11496 doesn't seem to be define in the database( WARNING : type of input 3 of step 11492 doesn't seem to be define in the database( WARNING : type of output 1 of step 11497 doesn't seem to be define in the database( WARNING : type of input 3 of step 11492 doesn't seem to be define in the database( WARNING : output 0 of step 11495 have datatype=1 whereas input 0 of step 11489 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21290050 AND mptpi.`type`=4209 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290051 order by id desc limit 1 Qualite : 0.04833780284360806 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21290052_11-03-2025_15_05_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290052 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21290052 AND mptpi.`type`=4207 To do Qualite : 0.0812687785847433 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21291428_11-03-2025_15_38_36.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21291428 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21291428 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21291429 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21291431 order by id desc limit 1 NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'11032025': {'nb_upload': 20, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1343022083, 1343022053, 1343021678, 1343021654, 1343021603, 1343021431, 1343021430, 1343021425, 1343021403, 1343021377] Looping around the photos to save general results len do output : 1 /21291428Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343022083', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343022053', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021678', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021654', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021603', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021431', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021430', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021425', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021403', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', None, '1343021377', None, None, None, None, None, '2658518') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.02097296714782715 save_final save missing photos in datou_result : time spend for datou_step_exec : 9.812116622924805 time spend to save output : 0.02126026153564453 total time spend for step 11 : 9.83337688446045 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 151.10user 151.41system 8:31.82elapsed 59%CPU (0avgtext+0avgdata 4230800maxresident)k 638424inputs+132944outputs (15364major+12575052minor)pagefaults 0swaps